ESP Manual

1. Overview

ESP (Experience Sampling Program) is free software for Palm Pilots to gather data from people in their daily lives. ESP displays questions, receives responses, and measures reaction times. It is useful for surveys, questionnaires, and scientific research.

In a nutshell:

1. You (the experimenter) hand out Palm Pilots to a bunch of people (the participants).
2. The Palms ask questions to your participants, recording their responses and reaction times. The set of questions can be asked as many times as you want. Each time is called a trial.
3. You upload the data to your favorite PC for analysis.

ESP is provided with no support. However, there is an ESP mailing list where you can discuss the software with other researchers. See section For More Information.

The development of this software was supported by National Science Foundation grant BCS 0322352 to Lisa Feldman Barrett.

About ESP

ESP for the Palm Pilot has two modes of operation:

• Automatic mode, which prompts your participants to answer questions at various times of day. This can happen at evenly-spaced times of day (a.k.a., fixed sampling) or at random times (random sampling).
• Manual mode, which lets your participants answer questions whenever they want. (This is sometimes called event-contingent sampling.)

Let's look at some examples.

1.1 Manual Mode: What it Looks Like

Manual mode is for event-contingent sampling, which is a fancy way of saying the participant launches the trials. Initially the Palm displays the screen:

At any time, the participant may tap the screen to begin a trial. She is guided through a sequence of questions, each with a set of response buttons at the bottom of the screen. For example:

When each question is answered, the next appears, until the last question has been answered. The participant is once again presented with:

and the process may be repeated.

1.2 Automatic Mode: What it Looks Like

Automatic mode is for fixed or random sampling, which means that ESP launches the trials itself. The participant is prompted to answer questions at times determined by the experimenter. Initially the Palm sleeps and displays a neutral screen:

When the time arrives for a trial, the Palm beeps (for a predetermined time set by the experimenter) and displays:

The participant has a limited time to respond (determined by the experimenter). If there is no response, ESP ends the trial and returns to sleep.

If the participant taps anywhere on the screen, however, she is then guided through a sequence of questions, each with a set of response buttons at the bottom of the screen. For example:

The participant once again has limited time to answer each question (again, determined by the experimenter). If she fails to respond to a question within the allowed time, the trial ends and ESP goes back to sleep. Otherwise, she responds to each question in sequence until the last question has been answered and the trial ends.

After a number of trials (set by the experimenter), the experiment ends, and the participant is shown:

1.3 Terminology

Packages

ESP 4.0 includes a suite of software packages:

• ESP, a Palm application that interacts with your participants.
• ESP Desktop, a PC application for designing experiments, configuring ESP settings, and collecting data. It runs on Windows and Linux. (A version for Macintosh OS X was planned originally, but proved too difficult for end-users to install.)
• ESPPrefs, an older Palm application for changing ESP settings. Because working directly on the Palm is inconvenient, we recommend you use ESP Desktop instead.

The program ESPView, supplied with older versions of ESP, is no longer needed nor recommended. Use ESP Desktop instead.

Experiments

The largest unit in ESP is called an experiment. It consists of:

• Questions: The stimuli presented to the participants. In ESP, these live in the ordinary Memo Pad application on the Palm. ESP reads the Memo Pad, finds the questions and displays them, and the participants respond.
• ESP settings: The behavior of ESP, as set by the experimenter; for example, how many times per day ESP should run, and whether questions should be in fixed or random order. The settings are set by running the Palm program ESPPrefs or the desktop program ESP Desktop.
• Data: Your participants' responses and reaction times. These are stored in a custom database on the Palm, and may be transferred to a personal computer via ESP Desktop.

People

The person who sets up ESP is called the experimenter. The experimenter runs ESP Desktop and/or ESPPrefs and designs questions. When we say "you" in this document, we generally mean the experimenter.

The people who use ESP are called the participants.

2. Installation

To run ESP in its full glory, you will need:

• One or more Palm Pilots or equivalent, to run ESP and ESPPrefs
• One desktop computer (Windows or Linux), to run ESP Desktop

Be prepared to dedicate your Palms and your desktop computer to ESP. This means:

• Don't use your Palms for other purposes. ESP uses the Palm Memo Pad for storing questions. If you create any other memos (unrelated to ESP) on your Palm, ESP may delete them. Also, ESP optionally takes control of the Palm, preventing any other applications from running. This is important because ESP measures the reaction times of your participants, so you don't want them distracted by other software.
• Don't use your desktop computer for other Palm-related purposes. ESP Desktop takes over some parts of the Palm hotsync process on your computer. It might not coexist well with other hotsync-related programs. Specifically on the PC, it replaces some of the standard Palm hotsync functionality (it installs new "conduits" into Palm Desktop that may prevent regular hotsynching from working). On Linux machines, ESP Desktop might not run correctly if other hotsync software is installed.

ESP and ESPPrefs should run properly on most Palms and compatibles. You should be thoroughly familiar with Palm usage, including:

• Hotsyncing and transferring data between the computer and the Palm, especially when multiple Palm Pilots are involved.
• Performing a soft reset (simple reboot, preserving data) and hard reset (major reboot, deleting all data) of the Palm.
• Deleting a database from the Palm Pilot. (If you want to conserve RAM on the Palm or uninstall ESP.)
• Graffiti. Simple use is needed for ESPPrefs (configuring ESP) and for entering questions in the Memo Pad. However, you can avoid Graffiti altogether by using ESP Desktop to run your experiments.

Now let's see how to install ESP.

2.1 Installing on Windows

Prerequisites

ESP has been tested on Windows XP. It will probably work fine on most other versions of Windows. Please tell us about any combinations that do or don't work.

Before installing ESP on a Windows machine, you must install:

• Mozilla or Firefox, the web browsers from http://www.mozilla.org, version 1.0 or higher. (Other browsers are not supported: we rely on custom features of Mozilla/Firefox.)
• Palm Desktop, the standard hotsynching software supplied with your Palm, or downloaded from http://www.palm.com.

Make sure these programs are installed before proceeding.

Installing ESP

To install this software, you will need administrator privileges on your Windows machine. Then follow these directions:

1. Make sure you have installed the prerequisites listed above.
2. If you haven't already done so, download ESP from the ESP web site. For Windows, ESP is provided as an installer program, EspDesktopConduitInstaller.
3. Double-click on EspDesktopConduitInstaller and follow the instructions. Some important notes:
   • Accept all the default values, or else ESP may not run properly. (For instance, don't change the default installation folder, `C:/Program Files/esp4', where all the ESP system files are kept.)
   • The installer might say you have to run a hotsync after installation. You don't have to do this. Just exit the installer.

   You have finished installing ESP. Now you must configure the Mozilla or Firefox browser to run ESP Desktop. This involves editing a Mozilla/Firefox system file to add one line.

4. Locate the system directory containing Mozilla or Firefox's system files. On Windows systems it is often `C:/Program Files/Mozilla' or `C:/Program Files/Mozilla Firefox'.
5. Locate the `chrome' subfolder and enter it.
6. Locate the file `installed-chrome.txt' and edit it with a plain text editor such as WordPad. (Don't use a word processor like Word or Wordperfect: you might destroy the file. Notepad also isn't great: it deals poorly with linebreaks.) If the file does not exist (which is the case in Firefox 1.5 and higher), create your own in the `chrome' subfolder.
7. Copy the line below and paste it into the file `installed-chrome.txt' at the very end, on a line by itself:

   content,install,url,file:///c:/Program%20Files/esp4/

   and press ENTER afterward.

   NOTE: Mozilla and Firefox each have their own `installed-chrome.txt' file. To use both browsers with ESP Desktop, you must paste the above line into both `installed-chrome.txt' files.

8. You're ready to run ESP Desktop. Run Mozilla and hit the URL:

   chrome://espdesktop/content/desktop.xul

   The first time you run ESP Desktop, you might see a warning about unsafe operation. See section Special note about permissions.

9. Test ESP Desktop by running its built-in self-test. Under the Help menu, select Test ESP Desktop and follow the instructions. See section Testing the Installation.

If the installation failed, or if ESP Desktop will not run, See section Troubleshooting. For a list of known bugs, See section Bugs.

Uninstalling ESP

1. In the Windows Control Panel, run Add Or Remove Programs, and uninstall every program whose name contains "ESP Desktop."
2. Delete the folder `C:/Program Files/esp4'.
3. (Optional.) Delete the ESP Preferences held within your web browser. Shut down all copies of Mozilla and/or Firefox. Then locate your browser preferences file `prefs.js', which is usually found somewhere in:
   • `C:/Documents and Settings/your_name/Application Data' on Windows
   • `~/.mozilla' on Linux

   Edit `prefs.js' using a plain text editor like Emacs or WordPad, not a word processor like Word or WordPerfect which will destroy the file. (Notepad isn't great either: it doesn't deal well with linebreaks.) Delete all lines that set a preference beginning with "esp.", such as:

   user_pref(esp.download.deletedata, true); user_pref(esp.latestexperiment, ...); user_pref(esp.questioneditor, ...); user_pref(esp.upload.esp, true); user_pref(esp.upload.espprefs, true); user_pref(esp.upload.initialize, false); user_pref(esp.upload.preferences, true); user_pref(esp.upload.preserveIDs, false); user_pref(esp.upload.questions, true);

   Then delete the lines that look like:

   user_pref(capability.principal.codebase.p0.granted, UniversalXPConnect); user_pref(capability.principal.codebase.p0.id, file://);

4. If you want to use this PC for other Palm hotsynching, reinstall your Palm Desktop software.
5. On your Palm(s), delete the applications "ESP" and "ESPPrefs."

2.2 Installing on Linux

ESP was developed and tested on SUSE Linux Professional 9.1. It will probably work fine on most other Linux distributions. Please tell us about any combinations that do or don't work.

Prerequisites

Before installing ESP on a Linux machine, you must install some other packages. Most Linux distributions include them, though they might not be installed by default, so check your computer and your installation media.

• Mozilla or Firefox, the web browsers from http://www.mozilla.org, version 1.0 or higher. (Other browsers are not supported: we rely on custom features of Mozilla/Firefox.)
• pilot-link from http://www.pilot-link.org/, a conduit library and toolset, version 0.11.8 or higher
• libxml2 from http://xmlsoft.org/, version 2.6.7 or higher
• A text editor such as emacs or vi

Make sure these programs are installed before proceeding.

Also, if you have other hotsync software running, such as KDE's kpilot, uninstall it. Otherwise kpilot will monopolize the USB port where your ESP Palm is connected, preventing ESP Desktop from finding your Palm. (Many KDE installations have kpilot enabled by default.)

Installing ESP

1. Make sure you have installed the prerequisites listed above.
2. If you haven't already done so, download ESP from the ESP web site. For Linux, ESP is provided in a compressed tar file, `esp4-linux.tar.gz'.
3. Extract the contents of `esp4-linux.tar.gz' into `/usr/local', e.g.,

   cd /usr/local tar xvzf <path>/esp4-linux.tar.gz

   When tar is finished, you'll see it has created a subdirectory, `/usr/local/esp4'. This is where all the ESP system files are kept.)

   You have finished installing ESP. Now you must configure the Mozilla or Firefox browser to run ESP Desktop. This involves editing a Mozilla/Firefox system file to add one line.

4. Locate the system directory containing Mozilla or Firefox's system files. On Linux systems it might be `/usr/local/mozilla', `/opt/mozilla/lib', `/usr/share/mozilla', etc.
5. Locate the `chrome' subdirectory and enter it.
6. Locate the file `installed-chrome.txt' and edit it with a text editor such as emacs or vi. (Don't use a word processor: you'll destroy the file.) If the file does not exist (which is the case in Firefox 1.5 and higher), create your own in the `chrome' subdirectory.
7. Copy the line below and paste it into the file `installed-chrome.txt' at the very end, on a line by itself:

   content,install,url,file:///usr/local/esp4/

   and press ENTER afterward.

   NOTE: Mozilla and Firefox each have their own `installed-chrome.txt' file. To use both browsers with ESP Desktop, you must paste the above line into both `installed-chrome.txt' files.

8. You're ready to run ESP Desktop. Run Mozilla or Firefox (as appropriate) and hit the URL:

   chrome://espdesktop/content/desktop.xul

   The first time you run ESP Desktop, you might see a warning about unsafe operation. See section Special note about permissions.

9. Test ESP Desktop by running its built-in self-test. Under the Help menu, select Test ESP Desktop and follow the instructions. See section Testing the Installation.

If the installation failed, or if ESP Desktop will not run, See section Troubleshooting. For a list of known bugs, See section Bugs.

Uninstalling ESP

1. Delete the directory `/usr/local/esp4' and its contents.
2. (Optional.) Delete the ESP Preferences held within your web browser. Shut down all copies of Mozilla and/or Firefox. Then locate your browser preferences file `prefs.js', which is usually found somewhere in:
   • `C:/Documents and Settings/your_name/Application Data' on Windows
   • `~/.mozilla' on Linux

   Edit `prefs.js' using a plain text editor like Emacs or WordPad, not a word processor like Word or WordPerfect which will destroy the file. (Notepad isn't great either: it doesn't deal well with linebreaks.) Delete all lines that set a preference beginning with "esp.", such as:

   user_pref(esp.download.deletedata, true); user_pref(esp.latestexperiment, ...); user_pref(esp.questioneditor, ...); user_pref(esp.upload.esp, true); user_pref(esp.upload.espprefs, true); user_pref(esp.upload.initialize, false); user_pref(esp.upload.preferences, true); user_pref(esp.upload.preserveIDs, false); user_pref(esp.upload.questions, true);

   Then delete the lines that look like:

   user_pref(capability.principal.codebase.p0.granted, UniversalXPConnect); user_pref(capability.principal.codebase.p0.id, file://);

3. If you uninstalled your previous hotsync software such as kpilot, reinstall it.
4. On your Palm(s), delete the applications "ESP" and "ESPPrefs."

2.3 Testing the Installation

After installing ESP on your desktop computer, you can check if it's properly installed. Run ESP Desktop and in the Help menu, select Test ESP Desktop. Follow the instructions and you'll get a report of success or failure at the end. If there's a failure, See section Troubleshooting.

If you cannot run ESP Desktop at all, then you probably forgot (or messed up) the step about editing the `installed-chrome.txt' file.

2.4 Using ESP without ESP Desktop

If you are a masochist, you can run experiments without ESP Desktop. Older versions of ESP worked this way. In this case:

• You'll need to install ESP and ESPPrefs onto your Palms manually, using Palm Desktop (Windows) or Linux software like kpilot or pilot-link.
• To edit ESP settings, you'll use ESPPrefs on each Palm. Alternatively, configure ESP using ESPPrefs on one Palm, copy the preferences to your PC, and then copy it back to your fleet of Palms one at a time.
• To edit ESP questions, you'll work directly in the Palm Memo Pad or Palm Desktop.

In short: take the time to use ESP Desktop. It is much more convenient.

2.5 Upgrading ESP

ESP 4.0 is more powerful than our old ESP 2 and its offshoot iESP (see section iESP) and includes important new features like ESP Desktop (see section ESP Desktop). However, ESP 4.0 is not completely compatible with the older versions. Your questions (the most complex part of your ESP experiment) are compatible, but other parts are not. The following table explains what is compatible and what to do about it.

2.6 Inventory

Here's a list of what gets installed with ESP.

Just about everything is placed into a single directory (folder):

Within this directory you'll find:

• `esp.prc', the ESP program for the Palm
• `espprefs.prc', the ESPPrefs program for the Palm
• `espprefs-conduit-unix' and/or `espprefs-conduit-win.exe', conduit programs used by ESP Desktop
• `espprefs-default.xml', the default settings file for ESP Desktop when you create a new experment
• `questions-default.txt', the default questions file for ESP Desktop when you create a new experment
• `experiment-default.txt', containing the default experiment name for ESP Desktop when you create a new experment
• Many files with names ending in `.xul' or `.js' or `.rdf', which are used by ESP Desktop

On Windows, several DLL files are also placed into `C:\Program Files\Palm':

• `EspDesktopConduit.dll', which controls hotsynching with an ESP Palm
• `DummyConduit.dll', which prevents a standard Palm hotsync operation with the Memo Pad

3. Creating Questions

The experimenter creates the questions that ESP asks during each trial. The same set of questions is used in every trial. However:

• You may randomize the order of the questions.
• You may define question sets and sample from them, using either ordered or random sampling.

3.1 Overview of Questions

ESP's questions are stored in the Palm's Memo Pad, one question per memo pad entry. You may create the questions in three ways:

• (Recommended) On another computer, using the ESP Desktop software package. See section ESP Desktop. In this case, you enter all your experiment's questions, one per line, into a text editor. ESP Desktop uploads them to the Palm, storing them in the Palm Memo Pad, where ESP will find and use them.
• On another computer, downloading them into the Memo Pad by hotsyncing, using the Palm Desktop.
• Within the Memo Pad, using the stylus and Graffiti.

You may create as many questions as you like, limited only by available memory.

Each question must be stored in a separate Memo Pad entry. Do not make the mistake of typing all your questions into a single memo on multiple lines. This won't work. You need to create a separate memo for each question.

Each question occupies one memo in the Memo Pad.

3.2 Ordinary Questions

Ordinary questions are the same type supported by the original ESP for the Palm. An ordinary question consists of five parts:

1. An integer ID number, either positive or negative, that uniquely identifies the question. Do not use zero.
2. The symbol "|" (vertical bar) which indicates the end of the ID number.
3. The question itself. It may contain any text except:
   • A vertical bar, "|", which is treated as a separator, as described later.
   • A percent sign followed by a letter, for example "%X", which is interpreted specially.
4. The symbol "|" (vertical bar) which indicates the end of the question text.
5. One or more labels for the response buttons at the bottom of the screen, left to right. Labels are separated by (you guessed it) vertical bars.

For example, the question "How are you feeling today?" with a unique ID of 37 and response buttons labeled "good", "bad", and "lousy" would be:

37|How are you feeling today?|good|bad|lousy

Notice that there is no additional space (i.e., space characters, tabs) surrounding the vertical bars, and there are no vertical bars at the beginning or end. Think of the vertical bars as separators: they go between items. Later versions of ESP are a bit more lenient about extra space, but for maximum compatibility, avoid it.

ESP will display this question as follows:

You may define as many button labels as will fit on one row of the screen. If you define too many, the screen will simply not look right (so be sure to examine each question's screen before running your experiment).

3.2.1 Response Types

By default, the responses for a question are presented as buttons in a horizontal row. You can change this with the %TYPE keyword.

37|How are you feeling today? %TYPE popup|good|bad|lousy

Supported %TYPE values are:

• buttons: One button per response. The default.
• popup: A popup list containing all responses.
• list: A selectable list displaying all responses.
• checkboxes: Checkboxes
• slider: A slider going from a minimum value to a maximum value
• text: Free-form text entry

3.2.2 About Question IDs

You must assign each question a unique, integer-valued ID between 1 and 32768, or between -1 and -32768. (Zero is not permitted.) These IDs serve two important purposes.

1. They are stored with the participant's responses. Therefore, in your data, you can identify which question is associated with each response. In this example question:

   37|How are you feeling today?|good|bad|lousy

   a "37" would be stored with the participant's response.

2. They may determine the order that the questions are asked in a trial. If you do not randomize the questions, they are asked in increasing numeric order by ID.

Your question IDs must be unique. If not, you will not be able to identify the question associated with each response.

3.2.3 Negative Question IDs

You may choose negative IDs for your questions (-1, -2, etc.). If you choose to randomize the order of the questions, negative IDs have a special purpose. Questions with negative IDs never get randomized: they are always asked in ascending order. Therefore, they always come before the randomized questions.

For example, suppose you specify the following questions:

1|Who?|a|b|c
2|What?|a|b|c
37|Why?|a|b|c
16|Where?|a|b|c
5|When?|a|b|c
-4|Are you ready to begin?|yes|no

If you do not randomize the questions, they will be asked in strictly increasing order: -4, 1, 2, 5, 16, 37. However, if you randomize the questions, the one with the negative ID ("Are you ready to begin?") will always be asked first. This allows your experiment to ask fixed, preliminary questions before beginning randomized questions.

3.2.4 What About Other Memo Pad Entries?

ESP notices only those Memo Pad entries that contain a "|" (vertical bar) in the text. Anything else in the Memo Pad is ignored by ESP. (So you could make notes to yourself if desired.)

3.3 Branching Questions with %NEXT

During a trial, by default, ESP presents its questions strictly in order by question ID. Sometimes you'll want to ask a question based on the answer to a previous question, e.g., "If the participant responds Yes, go to question 15, otherwise go question 71." ESP supports control of this kind using branching questions. (Branching was added by iESP.)

To create a branching question, add the keyword %NEXT to any response, followed by a question ID. For example, the question:

10|How are you feeling?|Good %NEXT 20|Bad %NEXT 30|Hungry %NEXT 40

branches to question 20, 30, or 40, if the participant responds Good, Bad, or Hungry, respectively. Here's a more fleshed-out example:

10|How are you feeling?|Good %NEXT 20|Bad %NEXT 30|Hungry %NEXT 40
20|I'm glad to hear that|OK %NEXT 50
30|Sorry to hear that|OK %NEXT 50
40|Would you like a doughnut?|Yes|No
50|Goodbye|OK

For example, if the participant sees question 10 and responds Bad, the next question displayed is 30, "Sorry to hear that." The participant responds OK, which branches to question 50, displaying "Goodbye".

Alternatively, if the participant sees question 10 and responds Hungry, the next question displayed is 40, "Would you like a doughnut?" Question 40 does not have any %NEXT keywords, so no matter how the participant responds, the next question will be the one with the next-higher ID, 50, "Goodbye."

You can also add %NEXT to the body of a question:

10|How are you feeling? %NEXT 200|Good|Bad|Hungry

which causes control to pass to question 200 after question 10, unconditionally.

%NEXT cannot refer to a question set ID, only a question ID.

This is illegal:
%QSET 1|100|Is this illegal?|yes|no %NEXT 101
%QSET 1|101|You cannot jump into a question set!|OK

3.4 Probabilistic Questions with %PROB and %SNEXT

By default, ESP asks all questions (unless you branch around them; see section Branching Questions with %NEXT). You can change this by assigning a probability to a question. (Probabilities were added by iESP.)

You assign a probability to a question with the %PROB keyword, like this:

10|How are you feeling? %PROB 50|Good|Bad|Hungry

The above question has a 50% likelihood of being asked. If it is not asked, it is skipped. When a question is skipped, control will pass to the "next" question in the experiment. This is ordinarily the question with the next higher ID.

10|How are you feeling? %PROB 50|Good|Bad|Hungry
20|What's your favorite fruit?|Apple|Banana|Cherry

In the above example, whether question 10 is asked or not, control passes to question 20. From our discussion of branching (see section Branching Questions with %NEXT), you know you can control the next question with the %NEXT keyword. But this applies only if the question is asked. What if it is skipped? You control this possibility with the %SNEXT keyword, which means "Skipped Next". For example, this question has a 75% probability of being asked:

10|How are you feeling? %PROB 75 %NEXT 30 %SNEXT 40|Good|Bad|Hungry

If asked, ESP will continue to question 30. However, if the question is skipped, the next question will be 40.

3.5 Question Sets with %QSET

ESP 4.0 has a new feature called question sets so you can do sampling of questions in a trial. Sampling may be ordered or random.

3.5.1 Defining a Question Set

A question set is simply a set of ESP questions, defined in the Memo Pad as usual, but each question is preceded by:

1. The string %QSET
2. Whitespace
3. A vertical bar

For example, if you have the following three questions:

10|This is the first question|OK
11|This is the second question|OK|Thanks
12|This is the third question|OK|Thanks|Goodbye

you turn them into a question set (say, with ID 5) like this:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye

These lines merely define a question set, but they do not use the questions in the experiment. They will not appear to any participants until you sample from the question set.

3.5.2 Sampling from Question Sets with %CHOOSE and %RANDOM

There are two ways to sample questions from a question set: ordered and random. Ordered sampling will use the questions in numeric order by question ID. Random sampling will choose random questions from the set.

Notice that the question set itself does not specify ordered or random sampling. It's just a set. The choice of ordered vs. random is made when you refer to the set, using a special question called a sampler. A sampler is a question that means "sample one question from a given set." Here is one example of a sampler that chooses a question from set 25:

100|%CHOOSE 25

Note that like other questions, this one begins with a question ID (100), but its question part is a sampler: the special keyword %CHOOSE, followed by the ID of the question set you're sampling.

All sampling is without replacement for a given trial. In other words, if question 16 is sampled during a trial, it will not be sampled again during that trial. If you have ten questions in a question set, and you sample it ten times, you'll get each question exactly once. If you sample from that set an eleventh time, that's an error in your experiment.

3.5.2.1 A limitation of %QSET

If you have a question set containing only N questions, your experiment may not sample from that question set more than N times. That's to be expected, since we sample without replacement, but there is also an associated bug: your experiment may not contain more than N questions that sample from that set. If you do, ESP may display the error "No questions in Question Set" and crash, even if the logic of your experiment would prevent sampling more than N times from your experiment.

For example, the following experiment will obviously cause trouble because it's sampling too many times:

%QSET 1|100|Sampled question|OK
1|%CHOOSE 1
2|%CHOOSE 1

There is only one question to sample, but you're sampling twice. That's clearly bad; but the next example is less obvious:

%QSET 1|10|Sampled question|OK %NEXT 999
50|%CHOOSE 1
51|%CHOOSE 1
999|Goodbye|OK

This experiment "should" work, but it doesn't. Although we sample twice from a question set with only one question, the second %CHOOSE should never run. That's because the first %CHOOSE will branch to question 999 and end the trial. However, because of the above-mentioned limitation, the experiment will fail.

We hope to eliminate this limitation in the future. In the meantime, if your experiment must include more %CHOOSE or %RANDOM statements than the size of your question set, there is a workaround. Define several identical question sets (with different IDs) and sample from them as needed.

3.5.2.2 Ordered Sampling from Question Sets with %CHOOSE

Ordered sampling is done with the %CHOOSE sampler keyword. Suppose you have a question set:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye

The statement:

%CHOOSE 5

means sample once from question set 5. Each time you apply %CHOOSE to a question set, the next question (in numeric order by ID) is retrieved and used.

This sampler itself is a question in your experiment, with its own ID number, say, 200:

200|%CHOOSE 5

Here is a complete experiment with the question set and several %CHOOSE statements:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye
200|%CHOOSE 5
201|%CHOOSE 5

This experiment will produce the following sequence of questions:

This is the first question
This is the second question

Here's another example:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye
200|%CHOOSE 5
201|How are you feeling?|Good|Bad|Hungry
202|%CHOOSE 5

which asks:

This is the first question
How are you feeling?
This is the second question

3.5.2.3 Random Sampling from Question Sets with %RANDOM

Random sampling is done with the %RANDOM sampler keyword. Each time you apply %RANDOM to a question set, a random question is selected and used from the question set. In the following experiment, two questions are selected at random from question set 5:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye
200|%RANDOM 5
201|%RANDOM 5

So the output might be any of the following outcomes:

This is the first question
This is the second question

This is the second question
This is the first question

This is the first question
This is the third question

This is the third question
This is the first question

This is the second question
This is the third question

This is the third question
This is the second question

Likewise, this example:

%QSET 5|10|This is the first question|OK
%QSET 5|11|This is the second question|OK|Thanks
%QSET 5|12|This is the third question|OK|Thanks|Goodbye
200|%RANDOM 5
201|How are you feeling?|Good|Bad|Hungry
202|%RANDOM{} 5

samples one random question, then displays "How are you feeling?", then samples another random question. Here's one possible result:

This is the third question
How are you feeling?
This is the first question

Remember, sampling is without replacement: no question will be repeated in a trial.

By default, random sampling always assigns every question in the set equal weight. That is, any question in the set is equally probable to be selected. You can change this with the %FREQ keyword.

3.5.2.4 Random Sampling with Weights from Question Sets, with %FREQ

The %FREQ keyword assigns positive integer weights to questions in a question set, representing the likelihood of being sampled. (Negative and zero weights are illegal.) For example, the following experiment that displays one question. That question is 25% likely to be the first, 60% likely to be the second, and 15% likely to be the third.

%QSET 5|10|This is the first question %FREQ 25|OK
%QSET 5|11|This is the second question %FREQ 60|OK|Thanks
%QSET 5|12|This is the third question %FREQ 15|OK|Thanks|Goodbye
100|%RANDOM 5

Weights need not sum to 100. The following example is equivalent to the preceding one.

%QSET 5|10|This is the first question %FREQ 5|OK
%QSET 5|11|This is the second question %FREQ 12|OK|Thanks
%QSET 5|12|This is the third question %FREQ 3|OK|Thanks|Goodbye
100|%RANDOM 5

To determine the probability of a question's being sampled, divide its %FREQ value by the sum of all %FREQ values in the question set. In the above case, the sum is 5 + 12 + 3 = 20, so the first question has likelihood 5/20 = 25%.

Some facts about %FREQ:

• Weights must be positive integers.
• %FREQ is used only for random sampling in question sets. It is ignored during ordered sampling, and when used outside of a question set.
• If you use %FREQ in a question set, you must assign %FREQ values to all questions in the set. You cannot mix weighted and unweighted questions in a single question set. If you mix them, your experiment has an error.

3.5.2.5 Sampling with Question Probabilities from Question Sets

You can use %PROB, %NEXT, and %SNEXT with question sets for both ordered and random sampling. But what happens in a case like this?

1|%CHOOSE 100 %NEXT 2
2|Hello|OK
3|Goodbye|OK
%QSET 100|4|My question %NEXT 3|OK

Here we have a sampler (%CHOOSE) with a %NEXT value, selecting a question ("My question") that also has a %NEXT value. Which %NEXT will win? The rule is this: the %NEXT value in the sampled question overrides %NEXT value in the sampler. So the above example will produce:

My question
Goodbye

In more detail:

• If the sampler has a %NEXT and the question does not, the sampler's is used.
• If the question has a %NEXT and the sampler does not, the question's is used.
• If both the sampler and the question have a %NEXT, the question's takes precedence.

The same rule applies for %SNEXT: if both the question and the sampler have it, the question's %SNEXT takes precedence.

3.5.2.6 Combining %FREQ and %PROB

%FREQ is not the same as %PROB, which is the probability that a question is asked at all (versus being skipped). You can use %FREQ and %PROB together just fine. If you use both in a single question:

• %FREQ is the probability the question will be selected from the set.
• %PROB is the probability that, once selected, the question will be displayed (versus skipped).

Consider this example:

%QSET 1|10|My question %PROB 44 %FREQ 25|OK
%QSET 1|11|Second question %FREQ 75|OK
888|%CHOOSE 1 %PROB 15

This means:

• Question 888 is 15% likely to get asked at all. This is the probability of DOING THE SAMPLING.
• If it is, then question 10 is 25% likely to be selected from the set, and question 11 is 75% likely.
• If question 10 is selected, then it's 44% likely to be displayed. If question 11 is selected, it will be displayed.

Thus, the probability of question 10 being displayed is (15*25*44)% and question 11's is (15 * 75)%.

3.5.3 Important Facts about Question Sets

• Question IDs and question set IDs are independent. Question 123 and question set 123 are unrelated and you can have both in the same experiment.
• All question IDs in an experiment must be unique. This includes question IDs inside question sets.
• Question sets cannot refer to other question sets, i.e., you cannot place a %RANDOM or %CHOOSE value inside a question set. This means you cannot create a hierarchy of question sets, one cascading into the next. Again, question sets are just boxes of plain questions.

  %QSET 1|100|%CHOOSE 2 This is illegal!

3.6 Custom Messages with %MSG

The special marker %MSG lets you change several text messages that ESP displays.

3.6.1 STARTTRIAL

When ESP begins a trial, it displays the message "Tap the screen to begin." You can change this message with the entry %MSG STARTTRIAL in the Memo Pad:

%MSG STARTTRIAL|OK, ready, let's go!

You should have at most one STARTTRIAL entry per experiment.

3.6.2 ENDTRIAL

When ESP ends a trial, it displays the message "Questionnaire completed. Thank you." You can change this message with the entry %MSG ENDTRIAL in the Memo Pad:

%MSG ENDTRIAL|All done now.

You should have at most one ENDTRIAL entry per experiment.

3.6.3 SLEEPING and WAITMSG

When ESP sleeps between trials, it displays the message "ESP is sleeping..." in the titlebar at the top of the screen. You can change this message with the entry %MSG SLEEPING in the Memo Pad:

%MSG SLEEPING|Snore snore...

By default, the rest of the display is blank. You can add a more detailed message below the titlebar using %MSG WAITMSG in the Memo Pad:

%MSG WAITMSG|The next trial will begin shortly. Please be patient.

You should have at most one SLEEPING entry per experiment, and at most one WAITMSG entry per experiment.

4. ESP Desktop

ESP Desktop is a major new feature of ESP 4.0! You can able to configure full experiments and manage all your participant data from your favorite personal computer. It runs under Windows and Linux.

ESP Desktop requires the Mozilla or Firefox web browser. Other browsers are not supported. (This is not just a whim: ESP Desktop is implemented in a programming language called XUL that works only in Mozilla-based browsers.)

4.1 Getting Started with ESP Desktop

Assuming you have installed ESP Desktop already on your local machine (see section Installation) here's how to run it:

1. Run Mozilla
2. Hit the URL chrome://espdesktop/content/desktop.xul

Now you're ready to work with an experiment.

Note: If you were previously working on an experiment, but you quit ESP
Desktop without closing the experiment, then ESP Desktop will offer you
the choice to reload that experiment.

To begin, you can either create a new experiment or work with an existing experiment.

4.1.1 Creating a new experiment

When you first run ESP Desktop, you'll be presented with this:

To create a new experiment, use the File menu and select New Experiment.

This action will display a dialog to pick a directory (folder) to contain your new experiment.

This is a standard Mozilla file dialog. Choose an existing directory, or create a new one, and click Select. (Note: the Mozilla file dialog is a little finicky about picking directories. You cannot select a directory by double-clicking it - if you do, you'll go into the directory. Instead, single-click the directory name and then Select.)

An empty experiment will be created in the directory you chose. ESP Desktop will display a default experiment name (usually "My New Experiment"), the file path to the experiment, and a new, extended menu bar:

and now you're ready to work on your experiment.

The first time you run ESP Desktop, you might see a warning about unsafe operation. See section Special note about permissions.

4.1.2 Opening an existing experiment

To open an existing experiment, choose Open Experiment from the File menu. This action will display a dialog to select the directory (folder) that contains your desired experiment.

The experiment will be loaded from the directory you chose. ESP Desktop will display the experiment name, the file path to the experiment, and a new, extended menu bar:

and now you're ready to work on your experiment.

The first time you run ESP Desktop, you might see a warning about unsafe operation. See section Special note about permissions.

4.1.3 Copying an experiment

To copy an experiment, any of these methods will work:

• Within ESP Desktop, open your original experiment, choose Save As, and save to a new location. This saves all experiment questions and settings, but not the participant data (but it's unlikely you want to copy that).
• Using your computer, copy the directory containing your original experiment. Optionally, in the copy, delete the contents of the `subject-data' directory, which contains the participant data for the experiment. (Unless you want to copy the participant data too.)

4.1.4 Special note about permissions

The first time you run ESP Desktop, you may see a warning dialog about unsafe operation:

This is because ESP Desktop needs to read and write files, and run other programs, on your computer. Web browsers don't normally do that. To use ESP Desktop, grant these permissions by checking the checkbox "Remember this decision" and clicking Allow. This eliminates the dialogs. If you don't do this, you'll see this ESP Desktop error:

and ESP Desktop will not operate.

Choosing "Allow" places the following line into your `prefs.js' file in your Mozilla folder.

user_pref("capability.principal.codebase.p0.granted", "UniversalPreferencesWrite UniversalXPConnect UniversalPreferencesRead");

which you can delete later if desired. (Shut down Mozilla before editing `prefs.js' or your changes won't take effect.)

A future revision of ESP Desktop will eliminate this issue.

4.2 Changing Settings

4.2.1 How ESP Desktop Operates

ESP Desktop is for editing the settings and questions for your experiment. Once you've modified them to your liking, you transfer them to one or more connected Palm Pilots.

You edit settings right in ESP Desktop's main window. Questions, however, are edited in a text editor (such as Notepad or Emacs) that ESP Desktop will launch for you.

When it's time to transmit your experiment between ESP Desktop and the Palm, here's what happens:

1. From ESP Desktop's menu, you choose a command such as "send to Palm" or "load from Palm"; there are a variety of such commands.
2. A dialog will appear, telling you to run a hotsync. The dialog looks different - and asks you to different things - depending on your operating system. If you're running Windows, the dialog looks like this:

   

   and is followed by:

   

   On the other hand, if you're using Linux, you get only a single dialog like this:

   

   In either case, do what the dialog says to do.

3. The desired information will be transmitted between your PC and your Palm.

So in general, ESP will prompt you to start a hotsync. Once it's done, ESP Desktop will be ready for your next action.

If things go wrong, you will see an error dialog. For example, if you fail to connect a Palm, you'll see

On Windows, if you click "OK" before a hotsync is complete, you might see:

4.2.2 General Settings for ESP Desktop

The Subject ID uniquely identifies the person carrying this Palm Pilot. If your experiment uses multiple Palm Pilots, make sure each of them is set with a different Subject ID. Valid values are 1 to 32767 inclusive. Do not use zero. Larger values may cause ESP to crash.

This value is used in two ways:

• It is saved with each of the participant's responses, so you can always identify which participant is associated with each response.
• It becomes part of the database name containing the participant's responses. [xxx] If the Subject ID is 123, the results database will be named ESPR-123 (on the Palm) or ESPR-123.pdb (on the Windows machine after uploading).

The Automation setting selects between manual mode (see section Manual Mode: What it Looks Like) and automatic mode (see section Automatic Mode: What it Looks Like) for your experiment. In manual mode, the participant initiates trials herself. In automatic mode, the Palm initiates them.

The Take Over Machine setting determines whether the participant can run other Palm programs while ESP is running. The recommended setting is to check this box. Otherwise, the participant will be able to halt ESP at any time and/or waste the batteries by running undesired software.

When we say "Take Over," we really mean it. None of the buttons will operate. This is good for keeping participants from fiddling with the Palm, but inconvenient when the experimenter needs to use the Palm, say, for transferring data to the PC using ESP Desktop. So, if you use Take Over Machine, you'll need to get the Palm out of this mode before you can hotsync your results data to your PC with ESP Desktop (see section Collect Data).

4.2.3 Automation Settings for ESP Desktop

These settings are available only if Automation is selected in General Settings. If you are running an experiment in manual mode (see section Manual Mode: What it Looks Like), these settings will not be visible when editing and will not take effect in the experiment.

The Begin Day and End Day settings specify the allowable times of day that trials may be run. Outside of this time range, no trials will occur. Times are on a 24-hour clock, from 0:00 (midnight) to 23:59 (11:59 pm). If the Begin and End times are equal, trials can run at all times. Setting the End time earlier than the Begin time is an error.

Total Trials and Trials Per Day control the frequency of trials. For example, if you request 100 trials in total and 20 trials per day, your experiment will last for five days (total trials divided by trials per day, or 100/20). If you request 101 trials and 20 per day, your experiment will last for six days, with the last day having only one trial (the hundred-and-first).

The Randomize Trials setting determines whether trials occur at predetermined or random times. If you've requested N trials per day, the period between the Begin and End times is split into N intervals of equal length. (For example, if the period is between 12:00 and 22:00 and you request 20 trials per day, this creates 20 intervals of 30 minutes each.) If Randomize Trials is unchecked, trials will occur at the beginnings of these N intervals. If checked, trials will occur at a random time within each interval. In our "20 trials per day" example, the first trial would occur at a random time between noon and 12:30, the next between 12:30-13:00, etc. So "random" means "random within each interval."

Max Seconds To Respond sets a time limit, in seconds, for the participant to respond to a trial. The Palm will beep continuously during this time. If there is no response within this time period, the trial ends. The related setting Seconds Between Beeps determines how frequently the Palm will beep while waiting for the participant to respond. (NOTE: Values of less than 2 seconds are unreliable.) So, if you request 30 seconds of beeping with 3 seconds per beep, the Palm will beep 10-11 times.

Run At Invocation is an experimenter's aid. When you first run ESP by tapping its icon, should a trial begin immediately (checked box) or should a trial simply be scheduled to run later (unchecked box)? Depending on your needs, you might want the participant to answer questions immediately (say, as practice in front of the experimenter) or not.

4.2.4 Question Settings for ESP Desktop

Max Seconds To Respond sets a time limit for a participant to respond to a question. If no response occurs within this limit, the trial ends. If you want no time limit on answering a question, use a large value like 99999999. (Or if appropriate, run the experiment in Manual mode instead of Automatic mode.)

Important: This timer begins running when a question appears, and it keeps running until the question is fully answered. If the participant slides a slider, writes Graffiti, taps checkboxes, etc., this does not prevent a timeout! In other words, the time limit can be reached while the participant is in the middle of writing Graffiti, moving a slider, etc. Be aware of this and plan appropriately.

Non-Answer Value sets the value to be stored if the participant does not answer a question.

Default Widget determines whether responses are presented as buttons, a dropdown menu, etc., by default. (If you don't want the default, you can specify the widget directly for each question with the %TYPE keyword.)

Randomize Questions determines whether the questions are presented in strictly ascending numerical order (based on their question ID: see section About Question IDs) or in random order. Remember that questions with negative IDs are never randomized (see section Negative Question IDs).