Osteoporosis is a bone disease associated with loss of bone density and prone to risk of fractures known as fragility fractures. Approximately, 8 million women and 2 million men ≥50 years in the US have osteoporosis.1,2 It is estimated that osteoporosis causes 1.5 million fractures annually.3 More than half of women and 25% of men ≥50 years are at risk of osteoporosis-related fractures. The projected cost for treating osteoporosis-related fractures by 2025 is estimated to be $25.3 billion.4 Hospitalization costs due to osteoporotic fractures among women ≥55 years were higher than those of myocardial infarction, stroke or breast cancer.5
Osteoporosis medications such as bisphosphonates increase bone density in postmenopausal women and thus reduce the risk of fragility fractures.6,7 However, medication adherence is low among osteoporosis patients. Approximately 50–70% of the patients discontinue their medication within the first year of medication initiation,8 increasing the risk of fractures in this population. The fragility fractures are associated with pain, disability, and even death in addition to substantial costs to society.9–13
A key group of patients to target for interventions are patients who have had a fragility fracture.14 Prior fragility fracture is a sentinel event that identifies the patient to be at high risk for future fractures. Some interventions directed at these patients and their providers have not been successful,9,15 but some interventions were successful. Martin et al recently conducted a systematic review including 32 published studies from 2003 to 2017.16 They classified interventions into three categories: health system (structural interventions), healthcare professional and patients. They reported that both structural interventions targeting the health system and interventions involving patients significantly improved the prescription of bone mineral density measurement, while healthcare professional-centered interventions did not have a significant impact on the prescription of bone mineral density measurement.16 Another systematic review and meta-analysis of randomized clinical trials of osteoporosis including 43 studies published between 2004 and 2017 indicated that several strategies appeared to be efficacious for improving bone mineral density testing and/or treatment rates in patients with recent or prior fracture.17 Those effective strategies included orthopedic surgeon or fracture clinic initiation for osteoporosis evaluation and treatment, multifaceted interventions targeting providers and patients, and patient education and activation.17 In another recently published systematic review of 55 randomized controlled trials, Kastner et al reported that multifaceted interventions including patient education, feedback, and follow-up significantly increased the initiation of osteoporosis medications.18 A systematic review of 57 articles with 64 interventions for the fragility fracture patients indicated <35% of patients initiated medication and <45% were medication adherent.14
The medication adherence was defined by the World Health Organizations in 2003 as “the extent to which the persons’ behavior (including medication-taking) corresponds with agreed recommendations from a healthcare provider.”19 Lindsay et al conducted a cross-sectional online survey with 1407 adult patients with osteoporosis, 35.7% had never been treated (n=503), and 23% had previously been treated (n = 323).20 They reported that the most common patient-reported barriers to not initiating or discontinuing osteoporosis treatment were treatment-related side effects and fear of side effects.20 Wozniak et al conducted interviews with 21 patients ≥50 years who had upper extremity fracture to understand factors that influence patients’ osteoporosis treatment decision-making.21 They reported three themes: negligible appreciation of risk regarding severity and impact of osteoporosis, ongoing evaluation of risks vs benefits of treatments, and re-evaluation of severity and impact of osteoporosis vs risks and benefits of treatment over time.21 There is limited information on the perceived barriers to treatment among patients with fragility fracture and barriers that are modifiable with intervention.21 Another qualitative study conducted with 37 postmenopausal women patients with osteoporosis and 18 physicians indicated that patients perceived barriers to treatment adherence included fear of side effects, absences of tangible results of treatment, view of postmenopausal osteoporosis, and severity of postmenopausal osteoporosis.22
According to cognitive-behavioral theory, patients’ perception and belief about their disease influenced their treatment behaviors. These perceptions either facilitate treatment adherence or serve as barriers.19 For Activating Patients at Risk for OsteoPOroSis (APROPOS) study, we developed an educational intervention for women with a history of fracture who self-reported as not currently receiving osteoporosis medication.23,24 In the current study, we 1) sought to understand patient-identified barriers to osteoporosis medication initiation, 2) aimed to understand which of these patient-identified barriers were potentially modifiable by a video intervention, and 3) focused on developing a cognitive map based on data that physician experts sorted patient-identified barriers. Cognitive mapping, a powerful visual strategy for organizing and communicating patient-identified barriers that will help us understand the complexity of osteoporosis treatment barriers and recognize the patterns and relationships from physicians’ perspectives. The University of Alabama at Birmingham’s institutional review board approved this study (IRB-110706002). Informed consent was obtained from all study participants. The guidelines outlined in the Declaration of Helsinki were followed.
Data Collection – Nominal Group Technique
We used nominal group technique (NGT) to identify patient perceived barriers.25,26 This technique has been used in multiple settings to identify problems faced by individuals with severe disability,27 develop a family intervention,28 define key domains for attending rounds,29 develop a decision aid to lupus treatment,30 facilitate organ donation,31 and develop tobacco control advice.32
Participants in NGT sessions were ethnically and racially representative of diverse postmenopausal women identified by their health care providers who had a history of prior fracture, were not on osteoporosis treatment, and were from two large cities, one from a Southern state and the other from a Western state in the US. The participants either had never taken osteoporosis medication or had previously taken but discontinued. Health care providers identified 18 postmenopausal women, 10 White, 5 Black and 3 Hispanic, who agreed to join the study from the two study sites. The average age of women NGT participants was 50 ± 7 years.
We conducted four NGT sessions, two at each location. The first two NGT sessions were conducted with 9 participants who had never taken osteoporosis medication, including five participants (3 White, 2 Black) in the Southern city site, and four participants (2 White 2 Hispanic) in the Western city site. The next two NGT sessions were conducted with 9 participants who had previously taken an osteoporosis medication but discontinued the therapy, including five 5 (2 White, 3 Black) in the Southern city site and four (3 White, 1 Hispanic) in the Western city site. All four NGT sessions were conducted by an experienced researcher (RS) with a research assistant. During each session, participants were briefly introduced about the purpose of the study and then were asked to respond to one question “What things make you not take or stop to take osteoporosis medication?” Participants were asked to write down their responses silently and independently and then were asked to present their responses to the group in a round robin format in order to have equal opportunity. A clarification process was followed to obtain common understanding from the entire group for each of the responses with no critique or evaluation. Each group generated a list of 26–30 perceived barriers. Then participants were asked to independently select and rank three that they perceived as the most important barriers that were not necessarily identified by themselves from the group list. The most important barrier was assigned 3 votes, the second important one was assigned 2 votes, and the third important one was assigned 1 vote. About 1/3 group generated responses were endorsed as important by participants in each group. Each NGT session lasted about 1.5 hours.33
In total, there were 37 responses that were ranked by participants from the four NGT sessions. A panel of four osteoporosis physician experts (JC, SM, KS, AW) independently reviewed the 37 ranked responses from the four NGT meetings and combined responses that had similar wording/meaning. This data reduction process resulted in 25 unique barriers that were used as the basis for card sorting and rating exercises by physician experts.
Data Collection – Card Sorting and Rating
Card Sorting and Rating Participants
A convenient sample of 27 osteoporosis physician experts with experience in treating osteoporosis from multiple institutions throughout the US was selected to perform card sorting and rating tasks. Lantz et al suggested that 10–15 card-sorting participants can provide optimal estimation of the similarity matrix,34 and other research indicated that 25–30 participants could yield results similar to those of several hundred when these participants are well represented and are familiar with the domain being considered.35 Among 27 physician experts, 17 were male (62.9%) and 25 were Caucasian (92.6%). Specialties included 11 rheumatologists (40.7%), 9 endocrinologists (33.3%), 3 general internists (11.1%), and 4 others with significant osteoporosis experience to consolidate concepts (14.8%). On average, physician experts had 26.6 ± 9.3 years of experience in treating osteoporosis, ranging from 8 to 41 years. More than half (56%) of physician participants reported greater than 50% of clinical work focused on osteoporosis.
Card Sorting and Rating
The 25 cards with one barrier on each card were sent to 27 osteoporosis physician experts. Physician experts were instructed to individually examine the barriers on the cards and group the barriers they felt were similar using their own personal criteria of how specific barriers fit together. They were asked to generate no more than 10 piles with each pile containing at least two cards. No themes or names were given to each pile. The physicians also used a five-point Likert scale, ranging from 1 (Strongly disagree) to 5 (Strongly agree), to indicate their level of agreement with the ability of each barrier to be addressed by a video-based intervention.
Data Analysis: NGT and Rating
Data from patient NGT sessions and physician rating survey were analyzed using descriptive statistics (eg, frequencies and percentage).
Data Analysis: Card Sorting
We analyzed data from the unforced card-sorting exercise using multidimensional scaling (MDS) analysis36 followed by hierarchical cluster analysis,37 to create a visual representation of the underlying dimensions of the barriers by placing them onto a cognitive map. We first constructed a 25×25 co-occurrence matrix from the piles created by each physician participant, and aggregated individual participants’ co-occurrence matrices for all 27 participants to create a group co-occurrence matrix. Numbers within this group matrix then represented the frequency that the two barriers were placed in the same pile. We then applied multidimensional scaling to the group co-occurrence matrix. Multidimensional scaling is an iterative process used to create an optimal geometric solution or “cognitive map.”36 This map is a spatial representation of the relationship between barriers: the relative proximity of any two points (each point representing one of the 25 barriers) is an approximation of the perceived similarity between those two barriers. The space itself can be unidimensional or multidimensional. The number of dimensions chosen to interpret a solution is based on the value of R-square and the corresponding stress statistic, as well as the overall meaningfulness and interpretability of the map. The closer the R-square is to 1.0, the better the map corresponds to the actual proximity data. In general, an R-square value >0.90 suggests high correspondence between the modeled solution and the observed data. The stress statistic is an estimate of the poorness-of-fit, with value >0.20 corresponding to an unacceptable fit and a stress value of <0.01 indicating a near perfect fit. Therefore, acceptable multidimensional scaling solutions typically have an R-square >0.90 and a stress statistic between 0.01 and 0.20. As the number of dimensions increases, the R-square value increases and the stress decreases. However, additional dimensionality often leads to interpretational difficulties. If more than one possible solution meets these criteria, then the most interpretable configuration is chosen.36 The MDS solution is interpreted by examining how barriers are arrayed along the dimensional axes relative to the meaning of the barriers anchoring each dimension.
We identified groups of homogenous barriers based on hierarchical cluster analysis, which is a technique used in concert with MDS to analyze data based on perceptions of similarity. The multidimensional scaling results include coordinates defining the location of each barrier within a derived multidimensional space and were used as data for a cluster analysis that was performed. The purpose of multidimensional scaling is to determine the relative ordering of the barriers ranked from more to less important, whereas the principal task of the cluster analysis is to assign the barriers into mutually exclusive groups. The results from the combined MDS and cluster analysis can be represented geometrically by a map reflecting different aspects of physicians’ perceived similarity of those barriers. In other words, barriers that were perceived as similar (ie, those frequently sorted together) were represented as points that are relatively closer together on the map than points further apart that were viewed as dissimilar. When MDS and cluster analysis are used together, it is possible to discern both the relative ordering of clusters and the ordering of individual barriers within each cluster along each dimension using rating data. The analysis was performed using SPSS v25 software (IBM SPSS Statistics, Armonk, NY: IBM Corp).
The 25 unique patients identified and prioritized barriers from NGT sessions are presented in Table 1 and were presented as an appendix in one paper.23
Table 1 Barrier Clusters and Levels of Agreement About Barriers That Could Be Addressed as Elements of a Minimally Invasive Intervention (n = 27)
Card Sort Results
The measures of overall goodness-of-fit indicated that a two-dimensional solution (RSQ = 0.96, stress = 0.10) was a better model than a one-dimensional model (RSQ = 0.80, stress = 0.26) and was comparable in fit and more interpretable than a three-dimensional solution (RSQ = 0.98, stress = 0.07). Research team members interpreted the results in two steps. We (RS, HQ, RO) preliminarily interpreted the results and then invited other team members (SS, KS) to discuss the initial interpretations. We interpreted one dimension (horizontal axis) as internal to the patients: patient uncertainty of care and provider trustworthiness and side effects of patients, anchored on the left by side effects of medications and on the right by patient uncertainty of care and provider trustworthiness. We interpreted a second dimension (vertical axis) represented as factors external to the patients such as access and complexity of treatment on the bottom and lifestyle change on the top (Figure 1).
The hierarchical cluster analysis revealed five distinct clusters to which each of the 25 important patient identified barriers were assigned an exclusive cluster membership. Because there are no assumptions that can be made about the distribution of the data in this analysis, a subjective decision was made by members of research team (RS, HQ) to interpret a five-cluster solution. The five clusters resulting from the hierarchical cluster analysis were superimposed on the multidimensional scaling map (Figure 1). The labels for each cluster were based on the commonality of all barriers included in the cluster. The five clusters are displayed on both Figure 1 and Table 1 were interpreted by all members of the research team.
Figure 1 This map reflecting different aspects of the perceived similarity of barriers is a geometric representation of the results from the combined multidimensional scaling and cluster analysis. The pairs of barriers perceived as similar are represented as points that are relatively closer together on the map than barriers that are viewed as dissimilar. The positive and negative numbers on the axes indicate only the location of the barrier and the distance between barrier clusters. Table 1 shows a listing of individual barriers in each cluster.
Table 1 shows the mean ratings for clusters and barriers within each cluster. The average ratings (mean ± standard deviation (SD)) of each of the five clusters in order of rating for modifiability were as follows: Cluster 1. Concerns about side effects (3.91 ± 0.91), Cluster 2. Experiences of side effects (3.68 ± 0.93), Cluster 3. Lifestyle changes (3.51 ± 1.17), Cluster 4. Medication access and complexity (3.38 ± 0.97), and Cluster 5. Patient uncertainty about treatment and trust in provider (2.72 ± 1.06) (Table 1). Within those 25 barriers, four were rated ≥ 4 on a 1 to 5 scale by physicians and were all classified in the “Concerns about side effects” cluster. The highest rating barrier was “Being told by my dentist that I could get bone and jaw cancer after taking the medication for a few years (4.08 ± 1.09)”, followed by three side effect-related barriers. Eleven barriers were rated between 3.5 and 4.0 by physicians including “Taking these medicines is complex and inconvenient (3.96 ± 0.82)” in the “Medication access and complexity” cluster, and “By making significant lifestyle changes to be healthier, it should not be necessary to take the medication (3.92 ± 0.93)” within the “Lifestyle changes” cluster. There were seven barriers rated with a mean score below 3 including “Not believing that my doctor is acting in my best interest (2.31 ± 1.01)” within the “Patient uncertainty about treatment and trust in provider” cluster (Table 1).
We engaged patients with a history of fragility fracture who were not receiving osteoporosis medication or who stopped taking osteoporosis medication to identify a wide variety of barriers/issues to initiate medication treatment and adherence. In addition, we sought insights from physician experts to understand how they view and categorize/organize patient-identified barriers that they believe may impact the initiation and adherence of medical therapy. By engaging both patients and physicians in this study about medication initiation barriers will facilitate the process of making shared and informed decisions toward osteoporosis treatment, and help researchers learned what kinds of barriers can be addressed by a video intervention and what kinds of barriers need to be addressed by other interventions in the future study.
Our cognitive mapping suggests that osteoporosis physician experts viewed patient concerns and experience of side effects as modifiable. This suggests that potential use of educational interventions by healthcare providers may assist patients to initiate and improve adherence to osteoporosis medication. In Lindsay et al’s study conducted in 2016 on barriers to osteoporosis medication, the second barrier was fear of side effects.20 The most common barriers that patients perceived toward osteoporosis treatment include drug-induced adverse side effects, pain, or other health risks.20,38–40 Patients must be educated by providers to manage the potential risk of side effects as compared to the greater benefit of an osteoporosis medication in reducing fracture risk.40–42 Educational interventions may also address the potential lifestyle changes required due to osteoporosis diagnosis.
Lifestyle changes were also reported as barriers to osteoporosis treatment in postmenopausal female patients with osteoporosis.20 Patients generally agreed with physicians’ recommendation about healthy lifestyle changes including dietary change and exercise, but they did not connect to the osteoporosis treatment because of worrying about safety of dairy foods and feeling difficulty of exercise.20 These are similar to the barriers within the “Lifestyle changes” cluster identified by patients from our study. Physicians also agreed that making significant lifestyle changes was a big barrier to patients (Barrier #3). However, interventions targeting barriers in patient lifestyle changes may be effective and doable.
Physicians sorted 5 patient-reported barriers into the fourth cluster, “Medication access and complexity.” There was an agreement between physicians and patients toward medication complexity (Barrier #20, #10, and #19) but not medication access (Barrier # 14 and #9). One study on medication decision with female patients with lupus reported that patients cared about affordability, costs, and resources to pay for their medication.30 The barriers related to medication costs and patient affordability may not be addressed directly. However, physicians should note the barriers to medication access when they prescribe medications to patients.
Some barriers such as the issue of trusting healthcare provider may be more difficult to modify as trust is dynamic over time and is influenced by education and information.43 In our study, “Not believing that my doctor is acting in my best interest” (Barrier #13 in the “Patient uncertainty about treatment and trust in provider” cluster) received the lowest mean rating score among 25 patient-reported barriers. This indicated a significant disagreement between physicians and patients. Further research is needed to examine whether physicians viewed modifiable patient-identified barriers can indeed be modified based on controlled trial, and to understand the individual barriers potentially associated with trust: eg, use of medical jargon, ability to communicate effectively about risks and benefits of medication.
Our study is the first to use a cognitive mapping approach to address patient-identified barriers in osteoporosis treatment initiation and adherence that could be used to develop educational interventions. The study represents a collaborative effort between patients with osteoporosis who were not receiving or adhering to osteoporosis medication and osteoporosis physician experts. The collaborative effort helps to design educational interventions that could be disseminated through community engagement efforts or through provider offices. The study used established methodological procedures and cognitive mapping approaches in a logically sequenced manner to construct a framework to understand patient-identified barriers to osteoporosis therapy. Cognitive mapping is a valid way to evaluate physician understanding of patients-identified barriers based on its substantive meaning and interpretability.
Our assessment of patient-identified barriers has several limitations. First, our data was based on a semi-empirical study with 18 patients and a card sorting review by 27 physician experts. Therefore, it is possible that our results may not be representative because of the small sample size of NGT sessions. Second, although there was considerable agreement between patients at the two study sites, the patients for the NGT were recruited from one Southern academic medical center and one Western private practice. These two sites may not be representative of all the different practice types in the US Third, as patient participation was voluntary; our results may be biased if patients electing to participate were different from those of patients who refused.
There is a great agreement among physicians on patient-identified barriers to medication initiation and adherence. The empirically derived model of barriers to medication initiation and adherence could be a guide to develop tailored interventions for improving osteoporosis treatment.
The identified clusters using a cognitive mapping approach provide a theoretical framework to develop future tailored interventions to address patient-identified barriers to initiation of or adherence to osteoporosis treatment. Having a clear understanding of the five domains of patient-identified barriers enables researchers to comprehend the complicated problem and to focus on those patient-identified barriers that are modifiable.
Dr Jeffrey Curtis reports grants, personal fees from AbbVie, Amgen, ArthritisPower, Aqtual, Bendcare, BMS, CorEvitas, FASTER, GSK, IlluminationHealth, Janssen, Labcorp, Lilly, Myriad, Novartis, Pfizer, Sanofi, Scipher, Setpoint, UCB, United Rheumatology, during the conduct of the study. Dr Susan Greenspan reports grants from NIH, grants from Amgen, during the conduct of the study. Dr Jeri Nieves received study drug from Radius and Eli Lilly for NIH trials, outside the submitted work. Mr Ryan Outman reports grants from NIH, during the conduct of the study. Dr Nelson Watts reports personal fees from Amgen, outside the submitted work. Dr Kenneth Saag reports grants from Amgen, grants from Horizon, LG CHem, Radius, and SOBI, during the conduct of the study; grants from Amgen, Horizon, LG Chem, Radius, and SOBI, outside the submitted work. The authors report no other conflicts of interest in this work.
1. Foundation. NO. Osteoporosis fast facts; 2015. Available from: www.nof.org.
2. Wright NC, Looker AC, Saag KG, et al. The recent prevalence of osteoporosis and low bone mass in the United States based on bone mineral density at the femoral neck or lumbar spine. J Bone Miner Res. 2014;29(11):2520–2526. doi:10.1002/jbmr.2269
3. Black DM, Rosen CJ. Postmenopausal osteoporosis. N Engl J Med. 2016;374(3):254–262. doi:10.1056/NEJMcp1513724
4. Burge R, Dawson-Hughes B, Solomon DH, Wong JB, King A, Tosteson A. Incidence and economic burden of osteoporosis-related fractures in the United States, 2005–2025. J Bone Miner Res. 2007;22(3):465–475. doi:10.1359/jbmr.061113
5. Singer A, Exuzides A, Spangler L, et al. Burden of illness for osteoporotic fractures compared with other serious diseases among postmenopausal women in the United States. Mayo Clin Proc. 2015;90(1):53–62. doi:10.1016/j.mayocp.2014.09.011
6. Imam B, Aziz K, Khan M, Zubair T, Iqbal A. Role of bisphosphonates in postmenopausal women with osteoporosis to prevent future fractures: a literature review. Cureus. 2019;11(8). doi:10.7759/cureus.5328
7. Eastell R, Walsh JS, Watts NB, Siris E. Bisphosphonates for postmenopausal osteoporosis. Bone. 2011;49(1):82–88. doi:10.1016/j.bone.2011.02.011
8. Jaleel A, Saag KG, Danila MI. Improving drug adherence in osteoporosis: an update on more recent studies. Ther Adv Musculoskelet Dis. 2018;10(7):141–149. doi:10.1177/1759720X18785539
9. Eisman JA, Bogoch ER, Dell R, et al. Making the first fracture the last fracture: ASBMR task force report on secondary fracture prevention. J Bone Miner Res. 2012;27(10):2039–2046. doi:10.1002/jbmr.1698
10. Solomon DH, Avorn J, Katz JN, et al. Compliance with osteoporosis medications. JAMA Intern Med. 2005;165(20):2414–2419.
11. Siris ES, Selby PL, Saag KG, Borgström F, Herings RMC, Silverman SL. Impact of osteoporosis treatment adherence on fracture rates in North America and Europe. Am J Med. 2009;122(2, Supplement):S3–S13. doi:10.1016/j.amjmed.2008.12.002
12. Imaz I, Zegarra P, González-Enríquez J, Rubio B, Alcazar R, Amate JM. Poor bisphosphonate adherence for treatment of osteoporosis increases fracture risk: systematic review and meta-analysis. Osteoporos Int. 2010;21(11):1943–1951. doi:10.1007/s00198-009-1134-4
13. van Oostwaard M. Osteoporosis and the nature of fragility fracture: an overview. Fragility Fract Nurs. 2018;2018:1–13.
14. Sale JEM, Beaton D, Posen J, Elliot-Gibson V, Bogoch E. Systematic review on interventions to improve osteoporosis investigation and treatment in fragility fracture patients. Osteoporos Int. 2011;22(7):2067–2082. doi:10.1007/s00198-011-1544-y
15. Giangregorio L, Papaioannou A, Cranney A, Zytaruk N, Adachi JD. Fragility fractures and the osteoporosis care gap: an international phenomenon. Semin Arthritis Rheum. 2006;35(5):293–305. doi:10.1016/j.semarthrit.2005.11.001
16. Martin J, Viprey M, Castagne B, et al. Interventions to improve osteoporosis care: a systematic review and meta-analysis. Osteoporos Int. 2020;31(3):429–446. doi:10.1007/s00198-020-05308-0
17. Nayak S, Greenspan SL. How can we improve osteoporosis care? A systematic review and meta-analysis of the efficacy of quality improvement strategies for osteoporosis. J Bone Miner Res. 2018;33(9):1585–1594. doi:10.1002/jbmr.3437
18. Kastner M, Perrier L, Munce SEP, et al. Complex interventions can increase osteoporosis investigations and treatment: a systematic review and meta-analysis. Osteoporos Int. 2018;29(1):5–17. doi:10.1007/s00198-017-4248-0
19. Sabaté E. Adherence to Long-Term Therapies: Evidence for Action. Geneva, Switzerland: World Health Organization; 2003.
20. Lindsay BR, Olufade T, Bauer J, Babrowicz J, Hahn R. Patient-reported barriers to osteoporosis therapy. Arch Osteoporos. 2016;11(1):19. doi:10.1007/s11657-016-0272-5
21. Wozniak LA, Johnson JA, McAlister FA, et al. Understanding fragility fracture patients’ decision-making process regarding bisphosphonate treatment. Osteoporos Int. 2017;28(1):219–229. doi:10.1007/s00198-016-3693-5
22. Alami S, Hervouet L, Poiraudeau S, Briot K, Roux C. Barriers to effective postmenopausal osteoporosis treatment: a qualitative study of patients’ and practitioners’ views. PLoS One. 2016;11(6):e0158365. doi:10.1371/journal.pone.0158365
23. Danila MI, Outman RC, Rahn EJ, et al. A multi-modal intervention for Activating Patients at Risk for Osteoporosis (APROPOS): rationale, design, and uptake of online study intervention material. Contemp Clin Trials Commun. 2016;4:14–24. doi:10.1016/j.conctc.2016.06.010
24. Danila MI, Outman RC, Rahn EJ, et al. Evaluation of a multimodal, direct-to-patient educational intervention targeting barriers to osteoporosis care: a randomized clinical trial. J Bone Miner Res. 2018;33(5):763–772. doi:10.1002/jbmr.3395
25. Miller D, Shewchuk R, Elliot TR, Richards S. Nominal group technique: a process for identifying diabetes self-care issues among patients and caregivers. Diabetes Educ. 2000;26(2):305–310. doi:10.1177/014572170002600211
26. Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks: Sage Publications; 2004.
27. Elliott TR, Shewchuk RM. Using the nominal group technique to identify the problems experienced by persons living with severe physical disabilities. J Clin Psychol Med Settings. 2002;9(2):65–76. doi:10.1023/A:1014931924809
28. Shewchuk RM, Franklin FA, Harrington KF, Davies SL, Windle M. Using cognitive mapping to develop a community-based family intervention. Am J Health Behav. 2004;28(1):43–53. doi:10.5993/AJHB.28.1.5
29. Roy B, Castiglioni A, Kraemer RR, et al. Using cognitive mapping to define key domains for successful attending rounds. J Gen Intern Med. 2012;27(11):1492–1498. doi:10.1007/s11606-012-2121-6
30. Qu H, Shewchuk RM, Alarcón G, et al. Mapping perceptions of lupus medication decision-making facilitators: the importance of patient context. Arthritis Care Res. 2016;68(12):1787–1794. doi:10.1002/acr.22904
31. Locke JE, Reed RD, Shewchuk RM, Stegner KL, Qu H. Cognitive mapping as an approach to facilitate organ donation among African Americans. Sci Prog. 2021;104(3):00368504211029442. doi:10.1177/00368504211029442
32. Qu H, Houston TK, Williams JH, et al. Cognitive mapping tobacco control advice for dentistry: a dental PBRN study. Am J Health Behav. 2011;35(2):228–239. doi:10.5993/AJHB.35.2.10
33. Delbecq AL, AVd V, Gustafson DH. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview IL: Scott Foresman and Company; 1975.
34. Lantz E, Keeley JW, Roberts MC, Medina-Mora ME, Sharan P, Reed GM. Card sorting data collection methodology: how many participants is most efficient? J Classif. 2019;36(3):649–658. doi:10.1007/s00357-018-9292-8
35. Wood JR, Wood LE. Card sorting: current practices and beyond. J Usability Stud. 2008;4(1):1–6.
36. Kruskal JB, Wish M. Multidimensional Scaling. Thousand Oaks, CA: SAGE Publications, Inc.; 1978.
37. Aldenderfer MS, Blashfield RK. Cluster Analysis. Sage; 1984.
38. Papaioannou A, Kennedy CC, Dolovich L, Lau E, Adachi JD. Patient adherence to osteoporosis medications: problems, consequences and management strategies. Drugs Aging. 2007;24(1):37–55. doi:10.2165/00002512-200724010-00003
39. Sheltawy AA, Criseno S, Gittoes NJ, Crowley RK. Fear of medication side effects is a barrier to optimal osteoporosis care. Osteoporos Int. 2015;26(2):843–844. doi:10.1007/s00198-014-2922-z
40. Warriner AH, Curtis JR. Adherence to osteoporosis treatments: room for improvement. Curr Opin Rheumatol. 2009;21(4):356–362. doi:10.1097/BOR.0b013e32832c6aa4
41. Cortet B, Bénichou O Adherence, persistence, concordance: do we provide optimal management to our patients with osteoporosis?. Jt Bone Spine. 2006;73(5):e1–e7.
42. Emkey RD, Ettinger M. Improving compliance and persistence with bisphosphonate therapy for osteoporosis. Am J Med. 2006;119(4):S18–24. doi:10.1016/j.amjmed.2005.12.019
43. Salt E, Rayens MK, Kerr AM, Alikhan M, Crofford LJ. Examining rheumatoid arthritis patients’ trust in their provider over time. Orthop Nurs. 2015;34(3):159–165. doi:10.1097/NOR.0000000000000145