How to use prototyping to improve the design of study enrollment procedures
MyHealthEd, Inc, in partnership with Power to Decide, Child Trends, and Healthy Teen Network, is working on a rigorous evaluation of the Real Talk app. One part of the study involves setting up a process for participants to enroll in the study that can be completed entirely online. This post shares how we used a human-centered design approach to prototype the design of study enrollment procedures.
Background
Designing a high-quality enrollment procedure is essential for ensuring we reach our recruitment goal and target population for the study. This is particularly important given that this study is taking place completely online. The process needs to be clear and easy to follow, as participants will not have a study representative to assist them. Enrollment is also the first of many activities participants will be asked to complete as part of the study, so it’s important that we create a positive experience for all participants. The full enrollment process for this study includes four main steps: Participants are required to (1) complete an eligibility screener, (2) provide informed consent to participate, (3) complete the baseline survey, and (4) download the study app to their phone.
To begin, we brainstormed all the different ways the four required steps could be ordered. We created user journey maps in which we considered the user experience as well as technical and data needs for each step of the enrollment process. Working through these initial planning activities enabled us to identify different ways to organize this process and the benefits and drawbacks for each option. Using a design thinking approach, we set up a short prototype to test which version of the enrollment process would most appeal to youth. The main goal was to gather real data from young people to better inform how the process would be received.
Our Approach
We designed the prototype to answer two questions: At what point in the enrollment process do we see highest attempts to download the app? And, where do we see the most drop off (i.e., where do we lose people in the process)?
We considered three different versions of the enrollment flow that included downloading the study app at different points in the process. We wanted to know how much information participants need before they are willing to download the app and how many steps they are willing to complete before they abandon the process. For this prototype, we simulated the steps of the enrollment flow as closely as possible through an interactive experience. We wanted to understand genuine user behavior as they navigated the enrollment process, rather than asking about attitudes and preferences in a more abstract or hypothetical scenario like an interview or survey.
We also wanted to spend few resources and learn quickly, so we designed a one-week test that could be run for $200. To do this, we used social media platforms to run multiple ads that directed people to one of the three enrollment scenarios (Figure 1). We created web pages and used free survey software, along with click-tracking links to capture the necessary information to answer our questions.
Figure 1.
Results
Results from this prototype helped us identify the version of the enrollment process that resulted in the highest attempts to download the app. We assessed user behavior in two ways. First, we calculated the overall download click rate for those who viewed the initial eligibility screener web page. Of those who viewed the initial page, 21.4% of those in Condition 1, 20.8% of those in Condition 2, and 4.0% of those in Condition 3 were willing to download the app.
Second, we tracked the download click rate among those who started the enrollment process (i.e., began entering information in the eligibility screener). Of those who began the screener, 92.1% of those in Condition 1, 75.6% of those in Condition 2, and 50% of those in Condition 3 were willing to download the app.
From this test, we gained valuable insight into participants’ willingness to download the app, and the amount of information they need before they are willing to do so. We were also able to simulate the entire enrollment flow with very simple and lightweight tools which resulted in data and helped us to identify next steps to move forward with designing the enrollment process.
Recommendations
Prototyping, like the process described here, is a valuable learning tool that can be used when developing programs and procedures to support public health research and programs. Below we provide three key recommendations for applying these strategies to other efforts:
(1) Consider implementing simple prototypes when designing research study procedures
Even when research studies take a participatory approach to designing programs, there is often less attention paid to how aspects such as the enrollment process are designed. Studies typically report on the evaluation process, including the ability to complete recruitment and reach a complete sample, the costs of implementing a study, and how data was collected. However, researchers rarely consider the user/participant perspective when designing other aspects of the study. Those who do seek participant feedback on study procedures typically describe these quality improvement efforts after a study has been implemented and/or completed rather than as part of proactive efforts to design user friendly study procedures. Prototyping offers an easy, low-cost way to make data-driven decisions to inform the design of study procedures.
(2) Be thoughtful and creative about what data is most useful for answering your learning questions
Since prototypes (by design) are limited in scope and breadth, it is crucial to consider what learning question(s) are most important and which may be secondary. Take time to identify what you hope to learn and what data you need to gather. This also includes being expansive with what data mean. Data do not have to be perfect in order to be informative and useful to answering your questions.
For example, first determine your general learning question. What is the overall aspect about which you want to learn more or gather data? As you design a prototype to help you answer this question, you can narrow your focus and ask more specific questions. For our process, our overall question was to learn more about user behavior and willingness to download the app. As we designed the prototype, we realized that we also wanted to track the data at every step to monitor drop off and thus took steps to track this data.
This test helped us determine whether and when participants were willing to download the app, but questions remain unanswered about what participants do after they download the app and whether they will complete the enrollment. This was outside the scope of the prototype – in other words, one test does not need to answer all of your questions – but focusing on specific components to test was valuable nonetheless to help us determine next steps.
(3) Engage in frequent monitoring throughout the test to identify any issues early in the process.
Frequent monitoring is important for making any necessary adjustments as you implement and refine the prototype. By thinking through the data and proactively identifying problems, we were able to make better sense of the prototype outcomes, track areas for improvement for future prototypes, and note new questions we’d like to answer.
Conclusion
Prototypes, and other human-centered design tools, can be incorporated into many different aspects of research, including when designing enrollment procedures. By using a human-centered approach, we gathered key information to tailor the enrollment process based on how young people engaged with the three different scenarios. These findings will inform the enrollment process for our study based on data from the user perspective.
We look forward to sharing more updates, stay tuned!
-Marisa & the Real Talk team
P.S. There are lots of ways to support our work further – whether through financial support or partnering with us to bring Real Talk to teens in your community. Email us with any inquiries at hello@myhealthed.org
