May - September 2018
Lab study @ Social Innovations Group at U of Michigan

Shipt: how might we leverage online shopping to help underserved communities in food desert

Methods

Field experiment, contextual inquiry, diary study, semi-structured interview

my role

Lead Researcher on a 2-person research team

STAKEHOLDERS

Principal investigator; academic communities

Summary

To uncover new opportunities for online food services to help underserved communities with food and transportation shortages, I led a mixed-method research study at the Social Innovations Lab at the University of Michigan. I conducted a field experiment consisted of home visits, interviews, and food item analysis to discover pain points around grocery shopping for urban communities. Strategies are proposed to better the user experience of online shopping from both a design and a policy perspective. (Study published at the ACM CHI conference in 2019 and won "Best Paper Award").

Context & Goals

From literature review, I learned that food and transportation access presented great challenges to urban communities who live in areas that lack resources, which may have subsequently contributed to a results of poor diet. In the meantime, the online grocery shopping model, which has great potential to break transportation barriers, is seen to be under-utilized by the exactly people who may need it the most.

To further investigate the "why" behind this series of phenomena presented in prior literature, I drafted 3 main research questions, hoping to understand if the online grocery shopping technology has the potential to help these communities, and if so, what design and policy implications need to be taken into account -

Selection of Shipt

At the time of the study, Shipt and Instacart were the only two online food delivery services available in the area (near Detroit, MI), and Shipt membership cost fit more within our budget. Therefore, I proposed that we use Shipt as a proxy platform to explore and evaluate the feasibility of such models, hoping that our research questions would be answered through participants' usage and experience on Shipt.

Methods & Rationale

The McGrath Wheel, on which I based my method selection decision, shows a complete categories of research strategies, each broken down into two methodologies. Research can never be perfect, so I was aware that whichever one I choose, it would be a tradeoff among generalizability, precision, or realism of the research.

Thinking back to my research questions: my goal is to understand the behaviors of a specific group of people, the underserved population, in a specific context - the grocery shopping experience. So, I value realism, or how much my research reflects reality the most, and I care the least about generalizing my findings to a larger population. Therefore, area C in the graph, field strategies, would be my area. In the end, I decided that a “field experiment” would be the best methodology, because it would allow me to conduct my research in a real setting in the field, but also have some degree of control by introducing different variables in an experiment.

McGrath's Wheel

Sample

I limited the sample to specifically underserved communities who lack transportation and fresh grocery access. To operationalize the criteria, I crafted 3 screening criteria:

  1. have limited access to transportation (self-reported as having challenges getting around)
  2. live in urban southeastern Michigan (a nearby "food desert" area near the research site)
  3. comfortable with technology (may be required to use Shipt on smart devices)

Study Design

The research questions require deep qualitative data into the experience of using Shipt (Research Question 1), it also requires a quantitative comparison to find out if there is a notable difference in health level of food consumption between those who use the service and those who don't (Research Question 2). Therefore, I decided to adopt a mixed-method model to include both qualitative (interviews) and quantitative (food item analysis in a field experiment setting). In addition, the experience cannot be analyzed as a one-time event as it wouldn't be realistic. I allocated 30 days and grocery funds for participants to experience the service in a realistic way and across time.

Key Results

RQ 1: Does the online grocery delivery model work to provide healthy-food access?

The study had shown that there are pros and cons when it comes to the current model of Shipt, specifically in regard to the particular needs of this population. Our participants found the online grocery delivery service to be very convenient, accessible, and reliable, which actually alleviated much pain from not being able to buy grocery freely without a car. However, an important pain point that surfaced was the incompatible payment method - many participants do not own a credit card and often buy food using EBT (electronic benefits transfer), and the Shipt payment system did not accommodate this, which can become a big reason for under-utilization of the service.


RQ 2: Does an online grocery delivery service lead to healthier food choices? 

Using the Australian Food and Drinks Classification Guide, I coded the items purchased by both groups based on healthfulness and calculated the difference between the group average over the 30-day period. No statistical inference was made due to the small sample size, but there is a notable difference between the two groups' purchasing choices in terms of healthfulness, descriptively. From user interviews, I hypothesized a potential reason could be that online shopping does not facilitate impulsive buying as much as in-person shopping.



RQ 3: What opportunities exist to support healthy choices among people living in low-income and transportation-scarce environments?

Synthesizing all the findings, I came up with 3 specific recommendations from both a design and a policy perspective to make the online grocery service at Shipt more accommodating and accessible for underserved communities: 


Impact

This project could propel real design change if done in the industry, and I f I were a UX researcher at Shipt…
  • I would recommend design solutions including inclusive payment method, partnership with local vendors, and more salient food labels for items in the digital products;
  • I would advocate for untraditional user groups such as the participants in this study to urge Shipt to become more inclusive to different types of users and expanding its current market segmentation.

The paper is published and won “Best Paper Award” at ACM CHI 2019 conference. It was well-received by the academic community and generated conversations around the transportation and grocery needs of underserved populations.

Learnings & Reflection

Working with vulnerable populations

When working with a specific segment of the population, it's important to keep in mind their unique challenges and needs during the research process. In this case, the resource-constrained population that I worked with faced transportation barriers, technology constraints, health issues, and more. These resulted in no-shows during onboarding and interview sessions, and unexpected drop-outs during the study. Although these added to the complications of the research, they can be better managed if I had anticipated some of the challenges they faced and the possible consequences for my study logistics.

Strategizing the recruitment process

When the research project has a specific target population, snowball sampling can be very valuable. During the onboarding process, I met a passionate participant Ben, a retired elderly who introduced me to an "insider's view" of the community. He shares with me the places members of his communities congregate most often, recommended places for me to post flyers and spread information, and even volunteered to help out with hand out flyers himself to his networks. I am still immensely grateful for his kind heart, which not only helped me gain insights into my target population but also added to the joy and empowerment I felt as a researcher!

End-to-end project management

I was fortunate to be given the autonomy to use the summer of 2018 to refine the study protocol and carry out the study on site. This project challenged me to plan a clear timeline and anticipate the time needed for each phase of the study while managing unexpected changes and act quickly throughout the process.

Checkout my other works