User and Usability Testing of a Web-Based Genetics Education Tool for Parkinson Disease: Mixed Methods Study

Introduction

Parkinson disease (PD) is the second commonest neurodegenerative disease and the fastest-growing neurological disease worldwide [1,2]. Variants in leucine-rich repeat kinase 2 (LRRK2), glucocerebrosidase (GBA), Parkin, Parkinsonism-associated deglycase (DJ-1), VPS35 retromer complex component (VPS35), PTEN-induced kinase 1 (PINK1), and α-synuclein (SNCA) are identified in 10%-12% of PD cases [3-9]. Despite a genetic mutation frequency similar to some cancer syndromes where germline genetic testing is common [10,11], genetic testing is not standard in the evaluation and management of PD and is rarely conducted as part of clinical care [12]. However, knowing one’s genetic status is already of key importance for research, as therapies targeting carriers of GBA and LRRK2 variants are in clinical trials [13,14]. Additionally, patients with PD have expressed interest in learning their genetic information [15,16]. Research programs such as the multisite PDGENEration study sponsored by the Parkinson Foundation (ClinicalTrials.gov NCT04057794) and the University of Pennsylvania (UPenn) Molecular Integration in Neurological Diagnosis (MIND) Initiative [17] have been developed to increase genetic testing and counseling.

Recently proposed recommendations would expand clinical or research genetic testing to nearly all patients with PD [18]. However, the standard service delivery model for genetic testing includes pre- and posttest genetic counseling, which is limited by the availability of specialized genetic counselors or physicians with sufficient genetics training [19]. Indeed, there are only 125 genetic counselors specialized in neurogenetics offering in-person visits, and 82 genetic counselors offering telehealth visits listed on the National Society of Genetic Counselors’ public directory for genetic counselors. We developed a web-based education tool focused on GBA and LRRK2 genetic testing called the Interactive, Multimedia Approach to Genetic Counseling to Inform and Educate in Parkinson’s Disease (IMAGINE-PD) to address this gap. IMAGINE-PD could be made available in research studies to increase genetics education prior to GBA and LRRK2 testing to identify research-eligible patients with PD.

The goal of this work is to create a genetics education tool for GBA and LRRK2 testing incorporating key stakeholder input. Structured interviews were conducted with movement disorder specialists (MDSs) and patients with PD to evaluate the content of IMAGINE-PD in user testing and with patients with PD to evaluate website usability according to the US Department of Health and Human Services (DHHS) Research-Based Web Design & Usability Guidelines [20]. Qualitative data analysis informed changes to create a final version of IMAGINE-PD.

Methods

IMAGINE-PD Website Development

Website development and usability testing were conducted in accordance with the DHHS research-based web design and usability guidelines [20]. Core content of a genetic counseling tool for GBA and LRRK2 variant genetic testing of PD was developed by the authors (TB, ACP, and TFT). GBA and LRRK2 variant testing information was included to align with the MIND Initiative, which is a whole-clinic biobanking effort at UPenn that offers optional genetic counseling and clinical confirmation testing [17]. Content was then assigned to “primary” or essential, “secondary” or important but not essential, or “optional” categories based on their level of importance according to expert input from genetic counselors and movement disorder physicians (RAP, TB, ACP, and TFT). Five primary concepts included a review of PD, concepts of basic genetics, genetics of PD, disclosure of genetic results, and limitations and implications of genetic testing. For each primary concept, an audiovisual recording of a movement disorder physician or genetic counselor describing the concept was created. Secondary information included diagnosis, symptoms, and treatment of PD, an introduction to genetic counseling, a review of genetic testing (GT), types of genetic tests and results, risks and benefits of GT, a review of GBA including Gaucher disease and LRRK2, and a review of other genetic causes of PD that are not tested in the study (limitations). One or more slides were created to capture the secondary content important for genetic counseling including text, visuals, and an optional audio recording of the text presented. Links to outside reading were identified for optional material. All content was organized into a high-fidelity prototype website using the Wix website [21]. Once the preliminary set of material was created, we solicited feedback on content and website organization from genetic counselors and physician-scientists with expertise in neurogenetics. Changes were made to address points of feedback. A new website was created with assistance from The UPenn Center for Clinical Epidemiology and Biostatistics Clinical Research Computing Unit. This new website included the content from the prototype and was made compatible with common web browsers and operating systems to be accessible on typical connection speeds across a range of different resolutions and orientations to account for use on a computer, smartphone, or tablet computer. Number of visits and time spent on each page are captured. A computer graphics specialist with experience in medical artwork created animations [22]. With this prototype, user testing was first conducted, and changes were then incorporated into IMAGINE-PD after reviewing feedback from patients with PD. These changes included content revisions on both primary and secondary topics. Videos with movement disorders and neurogenetics specialists were professionally recorded after implementing content changes from user testing. Readability of each page was evaluated using Readable [23] to ensure a Flesch-Kincaid readability index of below 9. Usability testing was conducted subsequently, and again feedback was reviewed, and refinements were made to the content, organization, and web interface. In the final version, the basic genetics video was broken into 2 videos for a total of 6 videos. Videos range from 51 to 92 seconds in length. A link was available below the video to read a transcript of the audio recording. The final version is hosted at UPenn [24]. A flowchart of the IMAGINE-PD web development and testing process can be viewed in Figure 1.

‎

Participants

In the user testing phase, 13 cognitively normal patients with PD who receive care at UPenn, referred by their physicians, were enrolled. MDSs at the UPenn Parkinson’s Disease and Movement Disorders Center and the Philadelphia Veterans Affairs Parkinson’s Disease Research, Education, and Clinical Center were asked to participate (excluding authors ACP and TFT) for a total of 11 neurologists or psychiatrists. In the usability testing phase, 12 cognitively normal patients with PD who receive care at UPenn, referred by their physicians, who did not participate in the user testing phase were enrolled. No patient or physician declined participation. All patients were previously enrolled in the MIND Initiative. In both phases, sample sizes exceeded the suggestions outlined in the DHHS usability testing guidelines [20]. Identified participants were approached in person, by phone, or by email. Informed consent was obtained, and study visits were performed in person by a single evaluator (NH).

User Testing

Structured interviews were conducted between a single evaluator with experience in conducting clinical research visits (NH) and a participant between October 31, 2019, and December 11, 2019. The evaluator neither had a prior relationship with any participant nor any assumptions or presuppositions about the outcomes of the interviews. First, participants were asked 11 questions about their general internet use, 4 questions about email and internet use for health information (adapted from Baker et al [25]), and 3 questions assessing background knowledge in genetics, PD, and genetic testing. During the interview, the evaluator asked questions of each participant, and data were entered directly into Research Electronic Data Capture (REDCap; Vanderbilt University) [26,27]. The evaluator navigated to each web page in the prototype allowing the participant to view and listen to all material on that page with unlimited time. Prompts were given to ensure all participants interacted with all aspects of each page. Feedback was solicited using 4 open-ended questions, a 1-10 rating of usefulness, and a 1-4 rating of clarity of presentation. After all web pages were reviewed, 7 additional questions were asked pertaining to the entire series of web pages and to solicit overall comments. Questionnaires are available in Multimedia Appendix 1.

Usability Testing

The second iteration of IMAGINE-PD was created after the results of user testing were reviewed, and changes were implemented. For usability testing, structured videoconference interviews were conducted between evaluator (NH) and participant between August 18, 2020, and September 14, 2020. Again, the evaluator neither had a prior relationship with any participant nor any assumptions or presuppositions about the outcomes of the interviews. First, participants were asked 8 questions about their internet use. The participant navigated to each web page with unlimited time to view all content. Participants were asked to use the “share-screen” function so the evaluator could observe their navigation of the website. Prompts were given to ensure all participants interacted with all aspects of each page. Feedback was solicited using 8 open-ended questions asked for each page. Questionnaires are available in Multimedia Appendix 1.

Statistical Analysis

Descriptive statistics are presented for all demographic data and scales. A qualitative content analysis plan was developed in advance of data review in consultation with Judy Shea, PhD. For both user testing review and usability testing phases, all data were collected into a spreadsheet excluding identifying information. Three evaluators (NH, RAP, and TFT) were given the following rules: (1) review the data from each web page for PD providers and patients with PD separately and identify 1 or more key themes per page, if a theme is apparent, and (2) if more than 1 theme is apparent, order themes for each page in order of importance or frequency. Data were independently reviewed by each evaluator to improve the trustworthiness of the analysis. Subsequently, the 3 evaluators met to establish a consensus on the key themes. Suggestions for changes to the website were made after all themes were reviewed and discussed among the evaluators.

Ethics Approval

Institutional review board (IRB) approval (UPenn IRB 834311) was obtained before initiating the study, and informed consent was obtained from all participants prior to any study activities. This research adheres to the principles set out in the Declaration of Helsinki.

Results

Website Development

The final version includes 27 web pages, including 3 video-only pages and 3 video pages with animations. Audiovisual pages also include accessible text for participants with hearing impairment. Each nonvideo page included audio recording of the text for auditory learners. The website is accessible by and formatted for computer browsers, mobile devices, or tablets. The length of time to view all videos is 7 minutes 42 seconds, and the estimated time to review all website content is 25-40 minutes. The Flesch-Kincaid reading level for all pages was a median of 8.3 (IQR 7.375-8.25) indicating all pages on the website maintained approximately an eighth-grade leading level. Screenshots of each web page are available in Multimedia Appendix 2.

User Testing

Participant demographics are summarized in Table 1. Email and internet use for health information is described in Figure 2A and Table S1 in Multimedia Appendix 1. PD, genetics, and genetic testing knowledge are reported in Figure 2B.

‎

For each web page, participants were asked to rate the usefulness (1 being the least useful and 10 being the most useful). The average usefulness across all pages for patients with PD was 8.49 (SD 0.65) and for MDS was 8.85 (SD 0.64). The results of usefulness for each page are found in Table S2 in Multimedia Appendix 1.

In user testing, content feedback pertained to the volume and level of detail of information (too much information), the complexity of the content (wording is too technical), and the focus of the content (make it more related to PD). Specifically, the amount of information was reported to be too much by patients on 13 occasions by 3 different physicians and 4 different patients pertaining to 11 different pages. Representative comments included “Gaucher's disease may be a little confusing. May not need to go into so much detail” and “alpha synuclein was confusing. Maybe too much content here.” On 6 different occasions, 6 different patients reported the topic, and the language used to be too technical including words like “bradykinesia” and “brain imaging.” Content was mentioned on 3 occasions by 2 different physicians and 1 patient pertaining to 3 different pages. Representative comments included “focus more on PD related genetic testing,” and “mention these are two genes we are looking at and this is why it is related to PD.” To address these points, we made changes to focus and simplify the content and to use plain language at a Flesch-Kincaid reading level 9 or below.

Three domains of feedback were identified on user testing analysis: content, function, and appearance. These domains were consistent between patient and physician participants, and suggestions for change were made based on the combined review of both sets of responses. Changes were made on 20 pages in response to content feedback, 4 pages in response to function feedback, and 4 pages in response to appearance feedback. A summary of the user testing qualitative analysis and recommendations for change are shown in Tables 2-4.

At the conclusion, participants were asked to provide summary feedback. A summary of responses is found in Table S3 in Multimedia Appendix 1. Comments regarding the order of the presented information, the web-based tools such as the buttons, links, and menus, and overall comments are provided in Table S4 in Multimedia Appendix 1.

Usability Testing

Participant demographics are summarized in Table 1. All participants reported using the internet or email within the past 12 months. Overall, 1 (8.3%) participant reported dial-up network use, 9 (75%) participants reported broadband network use, 7 (58%) participants reported using smartphones, and 8 (67%) participants reported accessing the internet via a Wi-Fi network. All participants reported using the internet to communicate with a health care provider, and 9 (75%) participants reported using the internet to search for health or medical information.

In usability testing, technical language was reported on 10 occasions, while the volume of information was reported on 6 occasions. Representative comments included “technical and a little difficult to understand” and “it was a little too much info in one video.” Based on usability testing feedback, further refinements were made as outlined in Tables 5-7. In addition to simplifying language, we included a glossary page, accessible via a link on every page, that defined key terms organized by concept. We also included a “Contact Us” page allowing participants to request additional information from a genetic counselor. Key points of feedback about website function included requests for a progress bar and mobile compatibility, which were both addressed for the final version. Improved audio quality was also suggested. Additionally, feedback on the difficulty of navigation through the website was mentioned by patients with PD and was addressed by creating clear instructions for navigation at the beginning of the website and large labels for website progression. Feedback about appearance included font color and size as well as picture choice. Final pictures were selected to ensure the representation of individuals of diverse race, ethnicity, age, and sex.

The same 3 themes were again identified on usability testing qualitative analysis: content, function, and appearance. Changes were made on 10 pages in response to content feedback, 6 pages in response to function feedback, and 7 pages in response to appearance feedback. A summary of the usability qualitative analysis and recommendations for change are shown in Tables 5-7.

Discussion

Principal Results

In this study, we evaluated a web-based genetics education tool for PD using evidence-based research methods to refine the content and conduct usability testing. First, content was developed based on expert opinion, and a high-fidelity prototype was created. Next, user testing was conducted through structured interviews with MDS and patients with PD to evaluate website content. Subsequently, usability testing was conducted via structured interviews with patients with PD. Using qualitative data analysis in both phases, we identified 3 domains of feedback (content, function, and appearance) and addressed feedback by incorporating changes to IMAGINE-PD to create a final version.

Comparison With Prior Work

Some studies in PD and Alzheimer disease have used alternative media forms as pretest education tools. For instance, the PDGENEration study, which offers genetic testing and counseling for PD, provides a pretest education tool that is a prerecorded video covering essential topics in PD genetics and genetic testing that was created by experts in neurogenetics and genetic counseling for PD (ClinicalTrials.gov NCT04057794). Additionally, the Alzheimer’s Prevention Initiative, which conducts apolipoprotein E testing in cognitively unimpaired people 60 years or older, uses a self-directed learning technique providing a brochure and a video covering content typically addressed in a pretest counseling session coupled with multiple-choice questions to reinforce learning [28]. To our knowledge, neither approach has undergone usability testing to incorporate the input of the end user as we demonstrate in this study.

Beyond neurogenetics, these results can also be viewed in the context of alternative genetic education tools, where more robust efforts to develop alternate education and disclosure methods are underway. A novel, web-based genetics education for a polygenic risk score of alcohol use disorders was evaluated in a randomized clinical trial of 325 college students. The tool was shown to improve user knowledge compared to general alcohol-use education alone [29]. In another study, a tool named Decision-Aid and E-Counselling for Inherited Disorder Evaluation [30] was developed to educate about genome-wide sequencing. Decision-Aid and E-Counselling for Inherited Disorder Evaluation was compared to pretest genetic counseling with a counselor; the genetics education methods were equivalent in conveying knowledge and were highly satisfactory to participants [31]. In the ongoing Communication and Education in Tumor Profiling and the Returning Genetic Research Panel Results for Breast Cancer Susceptibility studies, a web-based educational tool for pretest education was developed. These served as a guide for user testing and usability testing of IMAGINE-PD [32,33]. Furthermore, in the Study of an eHealth Delivery Alternative for Cancer Genetic Testing for Hereditary Predisposition in Metastatic Cancer Patients, web-based alternatives to traditional provider–mediated counseling and results disclosure are being evaluated. The outcomes of these studies will be informative for understanding the use of web-based genetic education tools even beyond inherited cancer syndromes. However, differences in the target populations, genetic testing performed, and the implications of the genetic test results between IMAGINE-PD and the Communication and Education in Tumor Profiling, Returning Genetic Research Panel Results for Breast Cancer Susceptibility, and Study of an eHealth Delivery Alternative for Cancer Genetic Testing for Hereditary Predisposition in Metastatic Cancer Patients studies necessitated the rigorous user testing and usability testing that we report here.

This study has several strengths that should be noted. First, the content for IMAGINE-PD was developed by experts in genetic counseling in PD, neurogenetics experts, and movement disorder physicians. Second, user testing included referring providers for neurogenetic services for patients with PD (movement disorder physicians) and the intended end user (patients with PD). Third, this evaluation followed the DHHS guidelines for user testing and usability testing, nearly doubling the recommended sample size in each phase for this type of research.

Limitations

Some limitations of this study should be acknowledged. First, the content of IMAGINE-PD is focused on targeted variant testing in GBA and LRRK2, limiting its scope and generalizability to other PD genetic testing. This was intentional to match the research-based GBA and LRRK2 screening being performed in the MIND Initiative at UPenn [17]. IMAGINE-PD may serve as the genetics education tool for MIND participants. Additionally, the MIND Initiative conducts the same genetic test for all involved study participants. As a result, the pretest education in the IMAGINE-PD tool deviates from a typical pretest genetic counseling session that would involve obtaining a family history and making decisions about test choices (family variant testing, multigene panel testing, exome, or genome sequencing). Instead, this is a scalable approach to screen everyone in the UPenn PD clinic for variants within the 2 most common genes associated with PD and identify potentially eligible participants for clinical trials enrolling carriers of variants in GBA or LRRK2. During a separate disclosure visit for GBA or LRRK2, a personal and family history could be reviewed in detail, and additional testing could be pursued afterward if indicated. Additionally, specialized content could be developed and added to subsequent versions of IMAGINE-PD to accommodate other specific types of diagnostic genetic testing. Second, all participants (physicians and patients) had a high level of education and experience with technology and computer use and interest in using a web-based pretest education tool, which probably does not capture the breadth of patients with PD and may overestimate the user experience. Ongoing evaluation of this tool will be necessary to determine which patients will be able to successfully use a web-based educational platform and who would be better served by other methods such as in-person or live telemedicine counseling with a genetic counselor. Although patients with PD were not involved in content creation, the content was derived from principals based on genetic counseling expertise. Patient feedback on content was solicited during both user and usability testing, where participants were given free answer choices to provide input on any additional topics they would want to have included. In the future, supplementary content could be developed to address deficits in patient or family-member comprehension, low literacy or education level, identifying potential psychosocial concerns to prompt additional counseling or comfort with technology. Making this tool accessible while in clinic on a smartphone, tablet, or computer screen may help to address limitations in access to technology. Third, participants elected to be enrolled in a genetic biobanking study and therefore may not represent the community with PD more broadly.

Conclusions

In summary, we present our findings from user testing and usability testing for the development of IMAGINE-PD, a web-based pretest genetic education tool. We describe a phased review and iterative process of refining the content, appearance, and functionality based on expert review as well as physician and patient feedback according to DHHS guidelines. The final version [24], which can be made available by request to the authors, will undergo further evaluation to compare it to standard telegenetic counseling with a genetic counselor (ClinicalTrials.gov NCT04527146) measuring satisfaction, impact, and comprehension. As a web-based learning tool accessible by internet, IMAGINE-PD has the potential to improve access to neurogenetic services for patients with PD interested in learning about their eligibility for LRRK2- or GBA-directed clinical trials.