May 2016 VOL 7, NO 4

← Back to Issue

Original Research

Development and Evaluation of a Technology-Enhanced Interdisciplinary Navigation Program for Low-Income Breast
Cancer Patients

Kathy J. Helzlsouer; Susan E. Appling; Susan Scarvalone; Shannon Manocheh; Ryan MacDonald; Lisa Gallicchio; Dawn Henninger; Arti P. Varanasi 

Background: Navigators have the potential to help overcome barriers that interfere with cancer treatment adherence, but their widespread use is economically challenging, especially in rural and low-resource communities. A centrally located interdisciplinary navigator team using technology to extend the navigators’ reach holds promise to efficiently improve treatment outcomes.

Objectives: To develop and evaluate a centralized “virtual” navigation program to support breast cancer patients through their adjuvant treatment.

Methods: The technology-enhanced navigator program was developed in collaboration with patient advisors and evaluated in a randomized clinical trial comparing use of the web-based navigation application with and without a navigator team (N = 98). A usability assessment was conducted at 3 months after enrollment. Comparisons by study arm were conducted using independent samples t tests for continuous measures and chi-square tests for proportions.

Results: Of the participants, 67% were minorities, mostly African American. Slightly more of the group randomized to the navigator team reported that the application was easy to use (77.6% vs 71.4%) and with higher confidence (71.4% vs 67.3%), but these differences were not statistically significant.

Discussion: A web-based interactive navigation program was developed in collaboration with breast cancer patients. Evaluation of the program demonstrated ease of use even among those with no prior computer experience.

Conclusions: A “virtual” interdisciplinary navigator program was confidently and easily used by low-income breast cancer patients who had a wide range in age, education levels, and prior computer experience.

Patient navigation in the cancer setting was first implemented to reduce disparities in screening and early diagnosis of breast cancer among disadvantaged populations.1-4 Navigational support has more recently been applied in the treatment setting with the goals of improving outcomes and quality of care, particularly as the complexity of treatment has increased.5-13 The success of treatment to decrease breast cancer mortality depends on the specific type of breast cancer as well as receiving the most appropriate treatment, such as chemotherapy, radiation therapy, and/or hormone therapy. However, poor patients are more likely than wealthier patients to have treatment delays or incomplete treatment, thus contributing to poorer survival.14-21 Missed appointments, poor symptom management, and limited psychosocial support contribute to poor adherence.15,21-29 Navigational support from an interdisciplinary team may help to overcome these impediments to receiving appropriate treatment.30

Proposed navigators include trained lay workers in addition to healthcare providers such as nurses or social workers. Providing personalized navigational services using trained healthcare providers can be economically challenging in low-resource communities. Centrally maintained services using technology to outreach “virtually” to patients may provide an economically feasible approach to providing personalized navigational services for cancer patients and improve treatment adherence among low-income patient populations.

The feasibility of such an approach is being evaluated in the Technology-Enhanced Navigation (TEN) Trial. The trial is designed to follow the suggested tenets for rigorously testing navigational programs31 and evaluate a virtual navigator approach with a primary focus on cancer treatment navigation rather than screening. This paper describes the development of the web-based navigation application, study design, conduct of the study, and the results of the usability assessment of the navigation application by trial participants.


Development of the Web-Based Navigation Application

The web-based navigation application was developed in collaboration with patient partners, a group of 5 low-income women who had been diagnosed with breast cancer and had completed adjuvant treatment. At the initial meeting, patient partners were asked to discuss their priority information needs during treatment, desired type of communication method, and to provide feedback on an initial prototype application.

Common themes and issues raised by the group were the sense of isolation experienced during treatment, the need for tailored information, and the importance of increased accessibility to reliable information. Among patients who had computer experience, a common issue was that searching the Internet on “breast cancer” produced an overwhelming number of sites, and, moreover, that many sites contained information that they knew was wrong or contradicted their doctor’s advice. Highly desired aspects of an intervention included being able to find targeted and reliable websites and having someone knowledgeable to talk to between appointments and assist in communicating with their doctors. With respect to the application interface, they wanted an easy-to-use and simple interface that did not require much typing.

A second meeting was held for a hands-on test of the online web-based navigation application tailored and developed based on their initial input. The women viewed the navigation application as a “medical tool” that should be part of their medical team. There was exuberant support for the application interface, which they found easy to use, even for those with no prior computer experience. The patients liked the use of simple icons, the access to tailored documents and videos to address their specific symptoms, and the ability to link to specific websites known to provide valid information. The “Tips & Resources” page (Figure) had icons connecting to resources in the following topic areas: common symptoms; nutrition; exercise; relaxation techniques; coping and feelings; work and family; practical resources; and treatment and beyond. Subtabs for vetted Internet links were located within each topic area and, for those who would receive navigation support, links to team-designed documents and videos. Internet links included websites with evidence-based approaches to cancer care information that were void of advertising. Videos, produced by the investigative team, included patient stories, food preparation tips and information on portion size, guided imagery, exercise, and an introduction to yoga.


The web-based application has a dual interface: one for the patient and an administrative interface for documentation by the navigator. The navigator is able to view the patient interface and to securely send and respond to messages via the application to trial participants.

Input into the trial design was also sought from the patient partners, who recommended that all participants receive a netbook computer with Internet access during the 1-year trial period and that all participants have access to the vetted Internet links. Thus, the comparison group had a similar application interface to the intervention group but only access to the vetted Internet links and no access to the navigator team’s videos or documents, or to a navigator.

The Navigation Team

To coherently address the variety of anticipated issues and problems that low-income women may face during their breast cancer treatment, an interdisciplinary team composed of a social worker, nurse, and medical oncologist was established. A study coordinator conducted the recruitment, enrollment, and computer training. For those randomized to the navigator arm, an assignment to either a nurse or social worker as the primary navigator was made within a week of enrollment. Assignments were made to balance patient load. Randomization to study arm was performed at the baseline visit. Weekly team meetings were held throughout the study, attended by the navigators, study coordinator, statistician, a medical oncologist, and technology leads. At each meeting, the navigators reported on their patients, and any unresolved issues were discussed and a plan formulated. Navigators were instructed to contact patients every 2 weeks or more frequently as needed. Virtual visits were documented in the Participant Log in the administrative section of the navigation application. Updates to the application based on navigator, coordinator, and participant feedback were discussed during the meeting and implemented as appropriate.

Study Design

The study design was a simple noncrossover randomized clinical trial comparing the technology-enhanced navigator program to Internet access alone. The usability of the application was assessed at 3 months with self-report questionnaires and review of data usage.

Patient Eligibility, Recruitment, Consent, and Randomization

Eligible patients were low-income, English-speaking women newly diagnosed with stage 0 to 3 breast cancer for whom adjuvant therapy with chemotherapy, hormone therapy, and/or radiation therapy was part of the recommended treatment plan. Recruitment was conducted through programs directed by the Maryland Department of Health and Mental Hygiene (DHMH) and through networking with social workers at hospital cancer centers and oncologists in private practices. At DHMH, recruitment was conducted in 2 ways through 2 different programs: the CDC-funded Maryland Breast and Cervical Cancer Screening Program (BCCP), which operates at the local health department level, and the Maryland Breast and Cervical Cancer Diagnosis and Treatment (BCCDT) Program, which is run centrally at the state level. At the local health departments, nurses identified women with newly diagnosed breast cancer through the BCCP and obtained the patient’s permission for study staff to contact them to provide additional information. Similarly, staff at the Maryland BCCDT Program, centrally located at DHMH, would identify patients and obtain their permission to be contacted. The Maryland BCCDT Program provides coverage for breast cancer treatment-related expenses for uninsured or underinsured women with incomes less than 250% of poverty level. The program is an “insurance” type program with coverage provided for costs related to breast cancer treatment, but with no case management. Recruitment was also conducted by outreach to providers throughout the state. Letters were sent to social workers, breast surgeons, and medical oncologists in the state to inform them of the trial. In-person presentations demonstrating the program were made to providers at their place of work and at cancer coalition meetings held in various county locations throughout the state. Approximately 30% of those enrolled were referred by the state program, and the remainder through other provider network referrals. Patients were recruited from all areas of the state with the exception of far western Maryland (Allegany and Garrett counties), where the contracted wireless provider did not provide adequate broadband service.

Potentially eligible patients were called by the study coordinator, given information about the study over the phone, and, for those interested, scheduled for an in-person enrollment visit at a location of their choice such as their home, local library, or doctor’s office. The majority of enrollment visits were conducted in the home. At the enrollment visit, after reviewing the study protocol, patients signed the consent, and the randomization envelope was opened, assigning the patient to either the navigator arm (intervention) or the Internet access–only arm (comparison). At the baseline visit, patients were trained in the use of the computer, provided a Quick Start Guide and a longer User Guide, and completed an online questionnaire. The visit ranged from 2 to 3 hours depending on the prior computer experience of the participant. All patients had a follow-up call from the study coordinator within 1 week of enrollment to answer any questions about access to the navigation application. Patients randomized to the navigator arm were contacted within 2 weeks by the assigned navigator. Patients were contacted either using Skype for videoconferencing or by telephone. Due to variation in speed of connectivity and difficulty in coordinating videoconferencing, telephone interaction became the favored mode of communication by the majority of patients and their navigators.

One patient was enrolled as a “run-in” for a final test of the navigation application. A total of 150 patients were screened for eligibility. Subsequently, 101 patients consented and were randomized: 51 to the navigation plus full navigation application access and 50 to the comparison arm with navigation application access only. One patient on the navigation arm was found to be ineligible shortly after randomization due to the presence of metastatic disease at diagnosis and was withdrawn from the study. Two patients, 1 on each study arm, withdrew immediately after randomization and before completing the baseline questionnaire. Thus, 98 patients, 49 per arm, were included in the analysis.

Data Collection

A health history form was completed at baseline that collected information on demographics, planned treatment, Functional Assessment of Cancer Therapy-Breast (FACT-B),32 health history (medical diagnosis and symptom checklist), and visual analog scales for pain, mood, sleep, distress, and fatigue.

A usability questionnaire was completed within the first 3 months of enrollment. Usability questions included Likert-style agreement statements such as: I use the application frequently; I think the application is easy to use; and, I feel confident using the application. To streamline the questions, the navigation application was referred to as a “portal” in the usability questionnaires. The usability questionnaire was developed based on the System Usability Scale, which is an industry standard for application usability testing.33 Additional questions asked users to rate the usefulness of the application features, and how often they used these features. A subset of questions was included specifically for those randomized to the navigator application intervention arm.

Participants were monitored for amount of data used through the contracted wireless service provider. Participants had 5 gigabytes (G) available per month, an amount adequate for access to the websites and for viewing of the videos posted on the application for those in the navigation arm. Participants were also instructed on how to use available free Wi-Fi services in order to avoid overages. The estimates of data use by the provider were rounded upward to the nearest whole digit number. For example, 300 megabytes, or 0.3 G, of data use was reported as 1 G of data use. Data were not available for estimating access to the application and data transfer through free Wi-Fi services.

Data Analysis

Comparisons of baseline characteristics, quality-of-life measures, and the usability assessment were conducted using independent samples t tests for continuous measures and chi-square tests for proportions. Adjustments for age and level of education were made using ordinary least squares regression for quality-of-life measures and ordered and binary logistic regression for usability assessment items.

Data usage through the wireless Internet provider was categorized as ≤1G, >1G and ≤3G, and >3G and compared between study arms with chi-square tests. The average data usage accessed through the Internet provider over the first 6 months of the study was compared between the 2 groups using independent samples t tests.


The baseline characteristics of the study participants are presented in Table 1. Those randomized to the navigation intervention arm were older (P = .04) and had a lower education level (P = .04) than those on the comparison arm. Although the participants on the navigation arm were more likely than those on the comparison arm to be unemployed, the difference was not statistically significant. The 2 groups did not differ in terms of race, marital status, body mass index, self-reported health rating, smoking history, and reported average alcohol intake.


At baseline, pain, fatigue, distress, mood, and sleep quality were not significantly different between the 2 groups (Table 2). Quality of life, assessed with the FACT-B, was similar between the 2 groups.


The results of the usability of the web-based navigation application in the first 3 months of the intervention period, by study group, are shown in Table 3 and Table 4. About half of the participants in each study arm reported that they used the application frequently (Table 3). The majority of participants in both arms reported that the application was easy to use, that they felt the various parts of the program worked well together, and that they felt confident using the application. Although the reported frequency of logging into the application was similar between the 2 groups (Table 4), those assigned to the navigator group reported accessing information more frequently than the comparison group (P = .004), and they were more likely to recommend the application to others (P = .04). The usefulness of specific features such as “Tips & Resources” and “My Schedule” and the reported comfort level in using the application were similar between the 2 groups.



Table 5 shows the estimated amount of data usage through the wireless Internet provider server during the first 6 months of the program. The wireless Internet usage was similar between the intervention and the comparison group.


Replacement of 6 computers was required due to the following reasons: 4 were replaced as a result of physical damage to the computer; 1 was lost or stolen; and 1 was replaced due to equipment failure.


The study includes patients ranging in age from 23 to 83 years at the time of enrollment with varying levels of education background and prior computer experience; all participants were low income based on requirements for the state insurance program and/or the published Housing and Urban Development guidelines for Maryland. The trial participants were good stewards of the equipment, with few replacement computers required. The results of the usability assessment demonstrate that a web-based navigation application, designed in collaboration with patients, resulted in an easy-to-use application that patients felt confident with and used often after only a brief training period. Data usage through the Internet provider did not vary significantly by study group, and trends over time suggest that most users accessed the Internet through their device most months of the study.

As noted, the majority of participants in both arms reported that the application was easy to use comfortably and confidently. Designing the application in participation with the patients likely contributed to usability of the application, and only a 2- to 3-hour training session was needed to assure confident and frequent use. The assessment of data usage through the Internet provider generally supported participants’ reported use of the computer. However, the assessment of data usage has limitations. In addition to access through the Internet provider, participants were instructed on how to use free Wi-Fi services, and the equipment was set to automatically access Wi-Fi where available. Therefore, among those who did not have any reported data usage through the Internet provider, it is unknown if they accessed through available free Wi-Fi services, through other devices, such as a home computer or phone, or simply did not access the application at all.

The acceptability and the ease of use of the web-based navigation application of the TEN program provides an alternative approach to standard in-person navigational services and thus holds promise for use in low-resource communities. Whether application access alone is sufficient to assure adherence to recommended treatment will be assessed by the trial. Use of technology for providing health information holds promise for improving health outcomes, but limited information is available on the application among low-income patient populations.34 Gustafson et al developed a patient support system, Comprehensive Health Enhancement Support System (CHESS), which incorporates extensive information services for breast cancer patients such as a frequently asked questions guide, a library of articles on breast cancer, a referral directory of resources, and web links.35,36 Short-term use of the CHESS system was higher among low-income women compared with more affluent women, and higher among rural low-income women compared with urban low-income African American women.35 The usability data from our study showed that a majority of the participants, all of whom were low-income, used the technology application and had confidence in their ability to use the application, consistent with the findings by Gustafson et al.35

In summary, an interactive “virtual” navigation program has been shown to be accessible and easy to use among a low-income patient population with a wide range in age and prior computer experience. The “virtual” personalized technology-enhanced navigation will be further assessed for its impact on treatment adherence and quality of life compared with information access alone.


  1. Freeman HP. Poverty, culture, and social injustice: determinants of cancer disparities. CA Cancer J Clin. 2004;54:72-77.
  2. Wells KJ, Battaglia TA, Dudley DJ, et al. Patient navigation: state of the art or is it science? Cancer. 2008;113:1999-2010.
  3. Ferrante JM, Chen PH, Kim S. The effect of patient navigation on time to diagnosis, anxiety, and satisfaction in urban minority women with abnormal mammograms: a randomized controlled trial. J Urban Health. 2008;85:114-124.
  4. Ell K, Vourlekis B, Lee PJ, et al. Patient navigation and case management following an abnormal mammogram: a randomized clinical trial. Prev Med. 2007;44:26-33.
  5. Early Breast Cancer Trialists’ Collaborative Group (EBCTCG); Clarke M, Coates AS, Darby SC, et al. Adjuvant chemotherapy in oestrogen-receptor-poor breast cancer: patient-level meta-analysis of randomised trials. Lancet. 2008;371:29-40.
  6. Clarke M, Collins R, Darby S, et al. Effects of radiotherapy and of differences in the extent of surgery for early breast cancer on local recurrence and 15-year survival: an overview of the randomised trials. Lancet. 2005;366:2087-2106.
  7. Early Breast Cancer Trialists’ Collaborative Group (EBCTCG); Correa C, McGale P, Taylor C, et al. Overview of the randomized trials of radiotherapy in ductal carcinoma in situ of the breast. J Natl Cancer Inst Monogr. 2010;2010:162-177.
  8. Early Breast Cancer Trialists’ Collaborative Group (EBCTCG); Darby S, McGale P, Correa C, et al. Effect of radiotherapy after breast-conserving surgery on 10-year recurrence and 15-year breast cancer death: meta-analysis of individual patient data for 10,801 women in 17 randomised trials. Lancet. 2011;378:1707-1716.
  9. Early Breast Cancer Trialists’ Collaborative Group (EBCTCG); Peto R, Davies C, Godwin J, et al. Comparisons between different polychemotherapy regimens for early breast cancer: meta-analyses of long-term outcome among 100,000 women in 123 randomised trials. Lancet. 2012;379:432-444.
  10. Cuzick J. Overview of adjuvant radiotherapy for breast cancer. Prog Clin Biol Res. 1990;354B:227-232.
  11. Cuzick J, Stewart HJ, Peto R, et al. Overview of randomized trials of postoperative adjuvant radiotherapy in breast cancer. In: Scheurlen H, Kay R, Baum M, eds. Recent Results in Cancer Research. Cancer Clinical Trials: A Critical Appraisal. Berlin, Germany: Springer-Verlag; 1988:108-129.
  12. Dowsett M, Cuzick J, Ingle J, et al. Meta-analysis of breast cancer outcomes in adjuvant trials of aromatase inhibitors versus tamoxifen. J Clin Oncol. 2010;28:509-518.
  13. Early Breast Cancer Trialists’ Collaborative Group (EBCTCG); McGale P, Taylor C, Correa C, et al. Effect of radiotherapy after mastectomy and axillary surgery on 10-year recurrence and 20-year breast cancer mortality: meta-analysis of individual patient data for 8135 women in 22 randomised trials. Lancet. 2014;383:2127-2135.
  14. Murphy CC, Bartholomew LK, Carpentier MY, et al. Adherence to adjuvant hormonal therapy among breast cancer survivors in clinical practice: a systematic review. Breast Cancer Res Treat. 2012;134:459-478.
  15. Budman DR. Dose and schedule as determinants of outcomes in chemotherapy for breast cancer. Semin Oncol. 2004;31(suppl 15):3-9.
  16. Wildiers H, Reiser M. Relative dose intensity of chemotherapy and its impact on outcomes in patients with early breast cancer or aggressive lymphoma. Crit Rev Oncol Hematol. 2011;77:221-240.
  17. Du W, Simon MS. Racial disparities in treatment and survival of women with stage I-III breast cancer at a large academic medical center in metropolitan Detroit. Breast Cancer Res Treat. 2005;91:243-248.
  18. Du Xianglin L, Gor BJ. Racial disparities and trends in radiation therapy after breast-conserving surgery for early-stage breast cancer in women, 1992 to 2002. Ethn Dis. 2007;17:122-128.
  19. Joslyn SA. Racial differences in treatment and survival from early-stage breast carcinoma. Cancer. 2002;95:1759-1766.
  20. Bettencourt BA, Schlegel RJ, Talley AE, et al. The breast cancer experience of rural women: a literature review. Psychooncology. 2007;16:875-887.
  21. Hershman DL, Unger JM, Barlow WE, et al. Treatment quality and outcomes of African American versus white breast cancer patients: retrospective analysis of Southwest Oncology studies S8814/S8897. J Clin Oncol. 2009;27:2157-2162.
  22. Griggs JJ, Culakova E, Sorbero ME, et al. Social and racial differences in selection of breast cancer adjuvant chemotherapy regimens. J Clin Oncol. 2007;25:2522-2527.
  23. Griggs JJ, Culakova E, Sorbero ME, et al. Effect of patient socioeconomic status and body mass index on the quality of breast cancer adjuvant chemotherapy. J Clin Oncol. 2007;25:277-284.
  24. Bhargava A, Du XL. Racial and socioeconomic disparities in adjuvant chemotherapy for older women with lymph node-positive, operable breast cancer. Cancer. 2009;115:2999-3008.
  25. Bonadonna G, Moliterni A, Zambetti M, et al. 30 years’ follow up of randomised studies of adjuvant CMF in operable breast cancer: cohort study. BMJ. 2005;330:217.
  26. Budman DR, Berry DA, Cirrincione CT, et al. Dose and dose intensity as determinants of outcome in the adjuvant treatment of breast cancer. The Cancer and Leukemia Group B.J Natl Cancer Inst. 1998;90:1205-1211.
  27. Chirivella I, Bermejo B, Insa A, et al. Optimal delivery of anthracycline-based chemotherapy in the adjuvant setting improves outcome of breast cancer patients. Breast Cancer Res Treat. 2009;114:479-484.
  28. Lenhart C. Relative dose intensity: improving cancer treatment and outcomes. Oncol Nurs Forum. 2005;32:757-764.
  29. Ruhstaller T, von Moos R, Rufibach K, et al. Breast cancer patients on endocrine therapy reveal more symptoms when self-reporting than in pivotal trials: an outcome research study. Oncology. 2009;76:142-148.
  30. Fiscella K, Whitley E, Hendren S, et al. Patient navigation for breast and colorectal cancer treatment: a randomized trial. Cancer Epidemiol Biomarkers Prev. 2012;21:1673-1681.
  31. Freund KM, Battaglia TA, Calhoun E, et al. National Cancer Institute Patient Navigation Research Program: methods, protocol, and measures. Cancer. 2008;113:3391-3399.
  32. Brady MJ, Cella DF, Mo F, et al. Reliability and validity of the Functional Assessment of Cancer Therapy-Breast quality-of-life instrument.J Clin Oncol. 1997;15:974-986.
  33. Brooke J. SUS: A “Quick and Dirty” Usability Scale. In: Jordan PW, Thomas B, Weerdmeester BA, et al, eds. Usability Evaluation in Industry. London, England: Taylor and Francis; 1996.
  34. Murray E, Burns J, See TS, et al. Interactive Health Communication Applications for people with chronic disease. Cochrane Database Syst Rev. 2005;(4):CD004274.
  35. Gustafson DH, McTavish FM, Stengle W, et al. Use and impact of ehealth system by low-income women with breast cancer. J Health Commun. 2005;10(suppl 1):195-218.
  36. Gustafson DH, Hawkins R, McTavish F, et al. Internet-based interactive support for cancer patients: are integrated systems better? J Commun. 2008;58:238-257.
Related Articles
Survivorship - April 29, 2016

A New Paradigm for Survivorship Care: Assessing the Patient’s Priorities

A new paradigm of survivorship care is needed that attempts to balance the patient’s total well-being against the often toxic treatment of the disease, suggests Deborah Korenstein, MD, Director of [ Read More ]

Evidence into Practice - April 29, 2016

Continuum of Care and Care Transitions: A Combined Conclusion from Novice and Seasoned Navigators

The role of the navigator along the continuum of care is bidimensional in nature with a patient-centered (empowerment with education and knowledge) and health system (multidisciplinary) orientation to deliver timely, [ Read More ]