Research Platform for Conducting Large Scale Field Experiments

Ziang Xiao
Hao Wu
Bo Zheng
Hari Sundaram
Crowd Dynamic Lab at University of Illinois at Urbana-Champaign

Today, one can run controlled studies on decision making in the lab, or large experiments on Amazon mechanical turk. Thus far, it has been nearly impossible to run field experiments at scale in locations where individuals take decisions. For example, how will customers react to a discount while browsing the aisles of a grocery store? Or react to advertisements while they shop? While lab experiments provide us with a noise-free gold standard, field experiments allow us to quantify the effects of receiving messages in the field while taking decisions.

We have developed a flexible, mobile-phone based experimental infrastructure that allows researchers without a computer science background to run large scale field experiments. Researchers configure an experiment page hosted on our web server, set location and time based triggers for tasks; recruited participants use a mobile application to participate. The experimenter need not write a single line of code: we host all experiments and store collected data on our server. The mobile application is universal in the sense that we can “push” each experiment to the same mobile application allowing a variety of experiments to be present in the same mobile application.

We have built the infrastructure and are evaluating the platform. Please contact [email protected] if you would like to use the platform for research.

Experiment Flow

On our platform, researcher is able to configurate an experiment through the web interface. Once reseracher publised the experiment, the experiment will be available to all users who installed our app on their phone. After the participant joined the study, he/she would receive notifications and respond to the tasks based on reseracher's setting. All data will be collected and recorded in a secure database that is only accessible to the resereacher.

Time Trigger

The time trigger allows the researcher to specify when and how often the data can be collected. Once the time trigger is triggered, the participant will receive a notification which will lead them to a task/message/survey specified by the researcher.

Location Trigger

The location trigger is trigger by a geological fence system. On the website, the researcher can mark an area on the map as a trigger and set up the condition for triggering the notification. The conditions include entering, existing or staying for certain minutes.

Experiment Market

To maximizing the exposure of the experiment and reach the most diverse sample, once the researcher published the experiment, the experiment will show up on the mobile app for all users. The researcher could set the criteria for targeting a specific group. If the researcher wants to recruit their participants, an invitation code can be set for accessing the experiment.

Randomization & Task Flow (Coming Soon)

Our research platform allows the researcher to randomize tasks and triggers depend on multiple stimuli which enables complex N by N experiment design. Similar to Quatrics’s survey flow, our task flow visualizes the flow of the experiment which aims to help the researcher to better manage their experiment.