Quality and readability of English-language Internet information for vestibular disorders
Abstract
BACKGROUND:
The Internet has become a powerful, accessible resource for many patients to use for their own medical management and knowledge. Vestibular disorders are prevalent, especially in the elderly. As the Internet is increasingly a major source of health-related information to the general public, it is often used to search for information regarding dizziness and vertigo. Ensuring that the information is accessible, unbiased, and appropriate can aid informed decision-making.
OBJECTIVE:
To evaluate the quality and readability of English-language Internet information related to vestibular disorders.
METHODS:
A cross-sectional website search using three keywords (nausea, dizziness, and vertigo) in five country-specific versions of the most commonly used Internet search engine was conducted in March 2018. The language was limited to English for all websites. Quality was assessed by presence of Health on the Net (HON) certification and DISCERN scores. Readability was assessed using the Flesch Reading Ease (FRE) score, Flesch-Kincaid Grade Level Formula (F-KGL), and Simple Measure of Gobbledygook (SMOG).
RESULTS:
In total, 112 websites were included and analyzed. The majority were commercial (61%) websites. A total of 42% had obtained HON certification. No association was found between the presence of HON certification and the resource of the website. The DISCERN scores had a mean of 2.52 (SD 1.1). Readability measures indicated that an average of 14–18 years of education was required to read and understand the Internet information provided regarding vestibular disorders.
CONCLUSIONS:
To ensure the accessible to the general population, it is necessary to improve the quality and readability of Internet-based information regarding vestibular disorders.
1Introduction
Health care models globally are encouraging patient-centered care by increasing public awareness of health-related conditions [49] and promoting self-management [27] through provision of health-related information [43]. The Internet is often used as the first source of health-related information [4] with more than two-thirds (69%) of health-related information queries being conducted though search engines such as Google [50]. As the Internet is used so frequently to search for information, comprehensive and unbiased information should be available to enable greater participation in care, and better-informed decision making by patients and their caregivers [16]. Concerns have been raised regarding the credibility of some health-related information presented online [3, 16]. This is partly related to the information presented not being controlled or needing to adhere to any ethical regulations [1, 42]. This may contribute to some of the health-related Internet-based information being biased and misleading [53].
In an attempt to promote trustworthy medical information on the World Wide Web, quality markers were developed relatively soon after the Web became publically available [58]. One such initiative is certification for reliable and useful online health information from Health On the Net (HON) Foundation. HON was founded in 1995 to encourage the provision of quality information that can be accessed by the general public [11]. Websites receiving a HON code certification indicate that the online information presented meets the ethical standard of offering quality, objectivity, and transparency of health information [9]. Although it is voluntary, it demonstrates the intent of a website to publish transparent information, uphold good practice guidelines and principles [11]. A further initiative to evaluate the quality of information provided on the Web is the DISCERN rating scale [13]. It consists of 16 quality indicators and provides a quantitative score to assess the health care information related to treatment decision-making. It provides a method to judge publication reliability, quality of treatment information, and an overall quality of information rating. While these quality markers exist, not all Web developers are aware of, or utilize them.
Ensuring the accessibility of online information in terms of ease of readability related to levels of comprehensiveness should be also be considered [2]. Information aimed at the general public should omit complex medical vocabulary and be at a reading level, which is easy to understand [5]. Readability refers to a measure of the difficulty experienced by people reading a text and is a measure of the linguistic characteristics of a given text [39]. Readability of written text is an objective measure of the level of reading skills an individual must possess to understand the material [4]. The most recent National Assessment of Adult Literacy [46] reported that the average American adult Reading Grade Level (RGL) was that of about seven years of education, though an even lower RGL was previously suggested for total comprehension [29]. Readability formulas are suggested to determine the ease with which materials can be read. These formulas analyze characteristics of the words or sentences in a passage and quantify the reading difficulty of the materials [24]. For most formulas, the estimate of readability is represented as a RGL, which can be interpreted as the number of years of US education required to understand what is written. Guidelines from the US Health and Human Services and The American Medical Association (AMA) recommends that health material should be written in plain language at or below the 6th reading grade level [57]. Monitoring that this is indeed followed for information provided on the Internet has more recently been evaluated. Although the readability of health-related information online has been evaluated in areas such as ophthalmology, dermatology, nephrology, orthopedics, psychiatry, and endocrinology [18, 19, 28, 30, 32, 44, 55] fewer studies in the area of audiology have been conducted [30, 32]. These have mostly related to the readability of websites related to hearing related disorders such as hearing loss, otitis media, and acoustic neuromas [35] or for tinnitus information [37]. No study has evaluated the quality, and readability of online vestibular information.
Due to the high prevalence of dizziness (17–30%) and vertigo (3–10%) [45], information related to these disorders are often sought [25]. The purpose of the current study was to examine the, quality, and readability of Internet information related to vestibular disorders. The study aims were to examine (a) the quality of the online vestibular information; (b) the readability level of online vestibular information; (c) whether readability and quality are associated to the source of the websites (i.e. commercial or academically).
2Methods
2.1Study design
A cross-sectional website search was used to identify websites aimed at the target population of adults with vestibular difficulties or their significant others. Ethical approval was not required due to the nature of this study.
2.2Search strategy
The keywords used were identified by firstly compiling a list of all the possible words related to vestibular disorders. Twenty people (7 hearing health professionals and 13 people affected by a vestibular disorder) were asked to provide 2-3 words they associate with vestibular disorders. Words such as vertigo, dizziness, nausea, migraine, BPPV, brain fog, cervicogenic, disorientation, vestibular disease, violent vertigo were identified (see supplementary material 1 for a full list). These words were compiled and run through a Google Trends search. To identify the three most commonly used words from this list, they were entered in Google Trends (www.google.com/trends), which compiles the relative frequency of key words in the search engine over time [25]. The three most frequent used words identified were selected as the keywords for this study, namely nausea, dizziness, and vertigo. The aim was to select a search strategy that would capture a global overview of English vestibular information websites. As Google is the most popular search engine at present, this search engine was used. When doing Google searches, Google automatically identifies the users Internet protocol and provides country-specific results. To ensure a that all the most frequently used websites were captured based on Google indexing, the top five English-speaking country-specific Google search engines (Top-Level Domains) were used, namely Australia, Canada, India, United Kingdom, and United States. The search was conducted during March 2018 by two of the authors (LP & BAF).
From the most frequently viewed English-language websites, the search was done in five country-coded Top-Level Domains. This stemmed in 15 separate searches (3 keywords X 1 search engines X 5 country-specific versions of the search engine). For each search, the first 20 websites (2 pages in Google) from the country-specific engine searches that met the inclusion criteria were included, as the majority of people do not explore more than two pages in Google [36]. Duplicate websites repeated in more than one country-specific search were excluded. The remaining websites were included in the analysis.
2.3Inclusion and exclusion criteria
Websites were included if they (1) were written in English (2) provided information about vestibular disorders such as vertigo, labyrinthitis or benign paroxysmal positional vertigo and their symptoms or treatment; and (3) were aimed at the target population of adults or their significant others with vestibular difficulties. Websites were excluded if they were identified as advertisements, news, images, and videos by Google. Two authors (LF and BAF) independently identified which of the websites met the inclusion criteria. The inter-rater agreement was recorded and analyzed using the Kappa statistic. Discrepancies were discussed, and if further discussion was required, a third author (VM) was consulted.
2.4Data extraction
For each included website, the website name, Uniform Resource Locator (URL) and the website’s source relating to the origin from where the website was created were extracted. [30, 32]. The website sources were categorized as follows [59]:
■ Commercial: websites that originate from an entity engaged in business activities such as developing, selling, or providing services or promoting a product for a fee.
■ Non-profit: websites that originate from organizations that are created for public benefit purposes, providing information about a subject without monetary benefit.
■ Governmental: websites from a national or state organization within a federal system responsible for the oversight and administration of specific functions, used restrictively by government entity.
■ Academic: websites that originate from an institute dedicated to education and research activities such as a university
■ Personal origin: websites created by an individual, not associated with any other organization, containing information not sponsored or regulated by any entity.
2.5Quality assessment
The following quality assessments were used:
1. HON certification [11] was used as a quality measure of reliability and credibility of information provided on websites. Verification of the presence of a HON certificate was determined by visiting the HON website https://www.hon.ch/) which lists certified websites.
2. DISCERN: The DISCERN instrument [13] consists of 16 items rated on a scale of 1–5 with higher scores indicating better quality. Ratings of 1 indicates that the quality criterion has not been met at all; rating of 2–4 indicates that the quality criterion has been met to some extent; and a rating of 5 indicates that the quality criterion has been completely met. Items 1 to 7 assess the publication reliability; items 8 to 15 assess the quality of information on treatment choice; and item 16 assesses the overall quality. Following studying and discussing the DISCERN handbook, one author (BAF) rated all the included websites and a second author (LF) rated 50% of the websites by random selection. Inter-rater agreement was calculated.
2.6Readability assessment
Readability of the website text was assessed using the Oleander Software [48]. As no standard for selecting readability formulas exists [12], a test battery of readability measures is generally recommended [4]. The formulae selected were those most commonly used to evaluated health-related information and those used in other audiology-related readability studies [34, 37] namely:
1. Flesch Reading Ease (FRE) [22]. This formula measures the grade reading level based on the average number of sentences and syllables per 100 words from at least three passages. It is scored between 0–100 with higher scores indicate higher readability The FRE is highly correlated with other readability measures [41].
2. Flesch Kincaid Reading Grade Level (F-K RGL). This formula was adapted from the FRE to translate the FRE score into an American school grade level, estimating the number of years of education required to understand the text. Lower scores indicate higher readability. The availability of F-K RGL as a Microsoft Word tool makes it convenient for health care professionals [31].
3. Simple Measure of Gobbledygook (SMOG) [39]. This formula uses the number of words included with at least three syllables to calculate an estimated grade reading level. It is based on 10 consecutive sentences selected from the beginning, middle and end of a text. Lower scores indicate higher readability. This estimates the years of education a person needs to understand a piece of writing. The SMOG represents one of the most valid readability measures [56] as it calculates Reading Grade Level based on 100% comprehension of the text. It is often preferred over the FRE and F-K RGL formulas, which may overestimate how readable a passage is as they are calculated at the level needed to comprehend 75% of the text [17].
2.7Data analysis
Data were analyzed using the IBM SPSS Statistics 24 software. In the first instance, descriptive statistics were explored, and assumptions of normality were tested. Where normality was not achieved data transformation (1/square root) was used if appropriate. Chi-square analysis was used to identify any baseline categorical differences in terms of the source of the website, and HON certification between the websites included. One-way analysis of variance was performed to ascertain any differences between continuous variables such as the DISCERN scores and readability measures. Where significant results were obtained, post-hoc testing using Bonferroni corrected t-tests for multiple comparisons were used. Pearson’s correlation was used to measure the strength between continuous variables such as the DISCERN scores and readability scores. The Spearman rank correlation was used to identify whether there was an association between the categorical or ranked variables such as the source of the websites and presence of a HON certificate. Significance at p < 0.05 was used to determine statistical significance for all statistical analyses.
3Results
3.1Websites included
In total, 112 websites were identified that met the inclusion criteria after removing the duplicates. The list of websites can be found in the supplementary materials.
The majority of the websites were from commercial sources (n = 68; 61%). Only a few were from non-profit organizational (n = 25; 22%), governmental (18; 16%), and academic (1; 0.9%) sources. The commerce behind the commercial websites included promoting drugs for dizziness or clinical services for those with dizziness. No websites were from personally uploaded sources. There were significant differences between the distribution of the three most common sources, i.e., commercial, non-profit, and government [X2(2) = 87.07, p < 0.001] as significantly more websites included were from commercial sources.
3.2Quality of the included websites
3.2.1HON certification
HON certificates were obtained by 47 (42%) of the included websites. The included academic website did not have a HON certificate. HON certification was present for 49% of the commercial websites and 28% both the non-profit organizational and governmental websites included. Although there was a significant association in obtaining and not obtaining a HON certificate, there no significant association between website source and HON certification [X2(2) = 2.04, p = 0.35] as seen in Table 1.
Table 1
Website quality indicators according to the HON certification and DISCERN scores
| Number (%) | Mean (SD) | Significant association (or difference)* | |
| HON certification | |||
| Yes | 47 (42%) | – | X2(1) = 5.99, p = 0.05 |
| No | 65 (58%) | ||
| HON certification by origin | |||
| Commercial | 33 (49%) | – | X2(2) = 2.04, p = 0.35 |
| Non-profit organization | 7 (28%) | ||
| Government | 5 (28%) | ||
| Academic | 0 (0%) | ||
| DISCERN score | – | 2.52 (1.1) | – |
| DISCERN score by origin | |||
| Commercial | – | 2.44 (0.9) | F(2,108) = 0.88, p = 0.42 |
| Non-profit organization | 2.76 (1.2) | ||
| Government | 2.44 (1.1) | ||
| Academic | 3.0 |
*Differences calculated between commercial, non-profit and government websites.
3.2.2DISCERN scores
As the total DISCERN scores were not normally distributed and were positively skewed, data transformation was performed to remove the skewness. When rating the different quality items, most of the individual items fell within a moderate score range (2–4). The highest DISCERN scores were achieved for stating the aims clearly, achieving the aims and relevance of the information provided (Table 2). The lowest DISCERN scores were related to how the treatment choice affects the overall quality of life, the areas of uncertainty and the risks of each treatment. The overall quality score was rated at 2.52 (SD: 1.1) which indicate potentially important but not serious shortcomings (moderate score of 2–4). For no item was a rating of 5 achieved (criterion has been complete met). For 7 items the criteria were not met (rating of 1). The inter-rater agreement for DISCERN was high as indicated by the interclass correlation coefficient of 0.93. No difference in DISCERN scores based on the source of websites were identified.
Table 2
Overall mean and standard deviations for the items of the DISCERN quality criteria for the 112 included websites
| DISCERN Items (Charnock et al., 1999) | Mean (Standard deviation) |
| Are the aims clear? | 3.86 (1.0) |
| Does it achieve its aims? | 3.58 (1.1) |
| Is it relevant? | 3.35 (1.1) |
| Is it clear what sources of information were used to compile the publication (other than the author or procedure)? | 2.42 (1.5) |
| Is it clear when the information is used or reported in the publication was reported? | 1.87 (1.2) |
| Is it balanced and unbiased? | 2.76 (1.1) |
| Does it provide details of additional sources of support and information? | 2.08 (1.2) |
| Does it refer to areas of uncertainty? | 1.72 (0.9) |
| Does it describe how each treatment works? | 2.17 (1.3) |
| Does it describe the benefit of each treatment? | 1.93 (1.2) |
| Does it describe the risk of each treatment? | 1.72 (1.0) |
| Does it describe what would happen if no treatment is used? | 1.97 (1.3) |
| Does it describe how the treatment choices affect overall quality of life? | 1.70 (1.0) |
| Is it clear that there may be more than one treatment choice? | 2.61 (1.7) |
| Does it provide support for shared decision-making? | 2.45 (1.3) |
| Based on the answers to all the above questions, rate the overall quality of the publication as a source of information about treatment choices. | 2.52 (1.1) |
3.3Readability
The readability measures are shown in Table 3, which suggest that on average 14–18 years of education is required to read and understand the websites focused on vestibular disorders. The majority of websites exceeded the recommended reading level of firth to sixth grades. A high percentage (over 80%) surpassed the average US adult grade level of seventh to eighth grades as shown in Table 3.
Table 3
Readability scores for website based on the origin and average grade levels
| Readability measure | Websites by Origin Mean (Standard deviation) Range | Differences between websites based on origin | Grade Level | |||||
| All websites | Commercial | Non-profit | Governmental | ANOVA results | Post-hoc comparison t-test | Exceeds 5th to 6th grade (recommendation) | Exceeds 7th to 8th grade (U.S. adult average) | |
| Flesch Reading Ease (FRE) Score | 49.25 (12.0) 18–100 | 49.23 (10.7) 18–77 | 42.68 (9.7) 28–61 | 57.5 (14.4) 31–100 | F(2,108) = 9.30, p = 0.001* | Commercial vs non-profit p = 0.04*; Commercial vs government p = 0.02*; Non-profit vs government p = 0.001* | 99% | 90% |
| Flesch Kincaid Reading Grade Level (F-K RGL) Formula | 9.67 (2.7) 4.7–19 | 9.39 (2.3) 5.2–16.4 | 10.89 (2.7) 6.8–16.3 | 9.21 (2.3) 4.7–19 | F(2,108) = 3.32, p = 0.04* | Commercial vs non-profit p = 0.51; Commercial vs government p = 1.00; Non-profit vs government p = 0.12 | 98% | 80% |
| Simple Measure of Gobbledygook (SMOG Score) | 11.1 (1.9) 7.7–16.5 | 11.15 (1.8) 8.3–16.5 | 11.41 (1.9) 8.9–15.2 | 10.62 (2.0) 7.7–15.9 | F(2,108) = 0.92, p = 0.40 | N/A | 100% | 99% |
Table 3, indicates where significant differences were present regarding readability scores based on the source of the website. No significant differences in SMOG scores based on the sources were found. Although the overall comparison for F-K RGL based on the source of the websites indicated a significant difference, no differences were found in the pairs contrasted during post-hoc testing. The overall comparison for FRE scores was significant. Post-hoc testing indicated significant differences between FRE scores for websites of commercial and non-profit, commercial and government, and non-profit and government.
The FRE score had moderate negative correlation with F-K RGL formula (r = –0.49) and also with SMOG (r = –0.48). The F-KGL Formula had a strong positive correlation with SMOG (r = 0.81).
3.4Association between quality and readability
The association between quality and readability of websites was assessed using the Pearson’s correlation test between DISCERN scores and readability measures. DISCERN had a small positive correlations with SMOG (r = 0.32) and with F-KGL Formula (r = 0.34), and no statistically significant correlation with FRE Score.
4Discussion
Vestibular disorders are common, especially in an aging population [45]. The Internet is increasingly used by the public to search for information regarding dizziness and vertigo. Ensuring that the information is accessible, unbiased and appropriate is important, but has not previously been investigated. The aim of this study was to assess the origin, quality, and readability of English-language Internet information available for vestibular disorders. The finding indicated that the websites were of moderate quality and that 14–18 years of education are required to read and understand the information, which is above guidelines of the 6th RGL [57].
The source of information may influence the type and quality of information included in the website. Thus, the source of the included websites was identified for the 112 included websites. Significantly more were of commercial of origin (61%), with a smaller selection from non-profit and governmental publishers and only one from an academic institution. These findings are consistent with previous studies focusing on audiology-based Internet information. Manchaiah et al. [37] identified 134 websites related to tinnitus information with the majority also being commercial of origin (49%). An earlier study assessing readability of Internet information for hearing impairment adults, identified 100 websites, again with most being of commercial origin (64%) [34]. As the majority of the websites were commercial of origin, their focus was on product promotion, selling or providing service. Encouraging more health-related websites from academic publishers, is important as previous studies have found that the accuracy of information is higher on academic websites in comparison with non-academic websites such as commercial and non-profit websites [40]. There are also indications that website ownership influences trust and credibility [52]. As the prevalence of dizziness and vertigo increases with age [8], many older people may seek further information regarding these conditions. Although older adults access online health-related information, they have a lower trust of such information [14]. Ensuring the availability of online information about vertigo and dizziness from sources perceived as credible is thus important.
Quality measures for this study were the presence of HON certification ratings using the DISCERN instrument. HON certification was found for 42% of the examined sites, indicating that most websites searched had no standardized quality certification. There were more HON certificates related to vestibular websites than those for tinnitus and hearing loss at 13% [34] and 14% [37] respectively. Although HON certification is increasing worldwide [9, 10, 21], it is still not present in the majority of health-related websites [26, 54]. This may be related to a lack of awareness regarding HON certification. More campaigning is required to raise awareness regarding this certification as this will aid the provision of accurate and unbiased information [50]. Of interest was that there was no association between HON certificate and the source of the website as also noted by Laplante-Lévesque et al. [34] for hearing impairment websites. In contrast, Manchaiah et al., [37] found that websites of government origin were more likely to have HON certification compared with those of other origins. Further studies should monitor this possible association.
Overall quality of the included websites was rated at was moderate at 2.52, when using the DISCERN scale. This slightly higher than that found for other audiological websites, such as those for tinnitus at 2.05 [37] and hearing of 2.39 [34]. Overall, these scores (score of 2–4 out of 5) do represent potentially important, but not serious shortcomings, indicating room for improvement. Considering that both the general public [15, 23] and practitioners [20] are increasingly relying on the information obtained online every effort should be made to promote presenting accurate, unbiased and complete data. Awareness of the DISCERN scale and desirable information to include on health-related websites should be promoted. The areas that had the lowest ratings were related to describing the risk of each treatment, referring to the areas of uncertainty, and how treatment choice affects the overall quality of life. This finding is useful for putting information online to ensure comprehensive information is included. Ensuring that websites are frequently updated can assist in ensuring the most up-to-date information is included.
It is imperative that health-related websites should be written at a level which can be easily comprehended by the general public. Many adults are known to have limited health literacy [6, 26, 33, 38]. The majority of websites in this study exceeded the recommended reading level of firth to sixth grades. A high percentage (over 80%) exceeded the average US adult grade level of seventh to eighth grades. This suggests that much of the information may not be accessible to the average adult. The results were consistent with other audiology-related studies finding similar results [34, 35, 37]. Ensuring that the language provided is accessible should be prioritized by web developers [30, 51]. The readability scores were exceeded as complex linguistic structures including the use of polysyllabic words were frequently present. It may be that the recurrence of words can potentially enable health literacy by stimulating the comprehension of the concept and introducing the jargon or polysyllabic words into the reader’s vocabulary. The only way to accurately understand the effect of repeating polysyllabic or jargon words (such as dizziness or giddiness) on a reader’s health literacy is to get behavioral measures of comprehension. This could be incorporated into future studies. Results suggest that some of the FRE readability scores are dependent on website origin. This was not the case for F-K RGL and SMOG scores. As this association was not found previously for tinnitus-related websites [32] if requires further investigation.
The DISCERN ratings were related to the SMOG and F-K RGL readability scores but not with FRE scores. These results suggest some association between higher quality and higher readability. This could be related to more medical terminology and longer sentences being included in more comprehensive websites. Of interest is that no association between quality and readability was found for hearing loss information presented on the Internet, possibly due to the variability of the quality [34]. The findings are also in contrast to those by Manchaiah et al. [37] who found a small negative correlation with quality and FRE and F-K RGL but no association with SMOG scores when investigating tinnitus websites. These findings are in contrast to those of the present study indicating an association between higher quality and lower readability. This may be related to the differences in information presentation between tinnitus and vestibular informational websites. Further research is required to establish whether readability is independent of quality or whether it is associated. Both factors should be considered during website development to ensure the information is comprehensive but easily comprehended by the general public.
4.1Study implications
Improving the readability of health-related Internet to be within the guidelines of being below the 6th grade level (add the reference: American medical association and U.S. Health and Human Services) is important to ensure health information is more accessible to the general population. Improving the quality of websites providing information regarding vertigo and dizziness is also important due to their characteristic complexities and various possible management strategies available. This includes having high quality information to address each type of common vestibular disorder such as labyrinthitis or benign paroxysmal positional vertigo, as their symptoms or treatment vary greatly. These differences often result in inaccurate perceptions about what may help, making it pertinent that websites address possible myths and misunderstandings together with providing accurate information. Achieving higher quality information online will contribute to the goal of improving health outcomes for those with vestibular disorders.
Clinicians should be aware of the general public reliance on the Internet for information regarding health care [36]. It is important to guide and direct patients to the most appropriate, accessible and comprehensive websites. These websites will aid the public gaining trust and credibility regarding information presented online. Factors know to contribute to online credibility such as the presentation of the content, the origin of the information, the website design, and layout should all be considered [52]. Web designers should also target populations that have a lower trust and credibility regarding information presented online, such as older adults [14]. Working at improving readability and quality of information aimed at these user groups is important.
4.2Study limitations and future directions
The current study has several limitations. The scope of this study was limited to English-language websites. Websites were excluded if they were non-English, and thus the results may not be applicable to a non-English-speaking patient population. A further limitation may be related to websites only being explored to the second page of the links, as generally performed in similar e-health information searches [59]. Although keywords were selected to represent the most common searches, it is possible that information seekers might use other terms. Use of other keywords could result in other websites being identified that may have different quality and readability outcomes than that of the websites included in this study. As the available resources on the Internet are always growing and changing, search results retrieved at different moments in time may differ. These results can serve as a comparative measure for reassessment in the future.
This study has focused on assessing the reading level of the materials provided. Reading level is not the only factor that affects comprehension. Further studies should focus not only on quality and readability but also investigate the accuracy and reliability of the information provided. In addition, the presentation style of the material may be an overall contributor to readability. Future studies should also asses the layout illustrations, message, information, and cultural appropriateness in conjunction with readability formulae. The User-Friendliness Tool (UFT) or Suitability Assessment of Materials (SAM) can be used to assess recommendations, such as the graphics, layout, typography, cultural appropriateness, and suitability [7].
5Conclusions
This was the first study evaluating the quality and readability of English-language Internet information for vestibular disorders. The majority of the websites reviewed were of moderate quality but were often not accessible due to high readability levels. It was encouraging that quality was not related to the source of the website (i.e., Commercial versus non-commercial websites). Future development of vestibular related websites should use these findings to ensure the general public have access to appropriate information, written in an accessible style.
Financial disclosures and conflict of Interest
None to declare.
Supplementary material
[1] The supplementary material is available in the electronic version of this article: https://dx.doi.org/10.3233/VES-200698.
References
[1] | Agarwal N. , Hansberry D.R. , Sabourin V. , Tomei K.L. and Prestigiacomo C.J. , A comparative analysis of the quality of patient education materials from medical specialties, JAMA Internal Medicine 173: ((2013) ), 1257–1259. |
[2] | Aldridge M.D. , Writing and designing readable patient education materials, Nephrology Nursing Journal 31: (2004), 373–377. |
[3] | Al-Jefri M. , Evans R. , Uchyigit G. and Ghezzi P. , What Is Health Information Quality? Ethical Dimension and Perception by Users, Front Med (Lausanne). (2018), Sep 20; 5:260. |
[4] | Badarudeen S. and Sabharwal S. , Assessing readability of patient education materials: current role in orthopaedics, Clinical Orthopaedics and Related Research 468: ((2010) ), 2572–2580. |
[5] | Beaunoyer E. , Arsenault M. , Lomanowska A.M. and Guitton M.J. , Understanding online health information: Evaluation, tools, and strategies, Patient Education and Counseling 100: ((2007) ), 183–189. |
[6] | Berkman N.D. , Sheridan S.L. , Donahue K.E. , Halpern D.J. , Viera A. , Crotty K. , Holland A. , Brasure M. , Lohr K.N. , Harden E. , Tant E. , Wallace I. and Viswanathan M. , Literacy and health outcomes, Evid Rep Technol Assess 199: ((2011) ), 1–941. Review. |
[7] | Biggs T.C. , Jayakody N. , Best K. and King E.V. , Quality of online otolaryngology health information, J Laryngol Otol 132: ((2018) ), 560–563. |
[8] | Bisdorff A. , Bosser G. , Gueguen R. and Perrin P. , The epidemiology of vertigo, dizziness, and unsteadiness and its links to co-morbidities, Front Neurol 4: ((2013) ), 29. |
[9] | Boyer C. , Baujard V. and Geissbubler A. , Evolution of health web certification through the HONcode experience, Stud Health Technol Inform 169: ((2011) ), 53–57. |
[10] | Boyer C. and Dolamic L. , Automated Detection of HONcode Website Conformity Compared to Manual Detection: An Evaluation, J Med Internet Res,17 (2015), e135. |
[11] | Boyer C. , Selby M. , Scherrer J.R. and Apple R.D. , The Health On the Net code of conduct for medical and health websites, Comput Bio Med 28: ((1998) ), 603–610. |
[12] | Breese P. and Burman W. , Readability of notice of privacy forms used by major health care institutions, JAMA 13: ((2005) ), 1593–1594. |
[13] | Charnock D. , Shepperd S. , Needham G. and Gann R. , DISCREEN: An instrument for judging the quality of written consumer health information on treatment choices, J Epidemoil Community Health 53: ((1999) ), 105–110. |
[14] | Chaudhuri S. , Le T. , White C. , Thompson H. and Demiris G. , Examining health information seeking behaviors of older adults, Comput Inform Nurs 11: ((2013) ), 547–553. |
[15] | Couper M.P. , Singer E. and Levin C.A. , et al. Use of the Internet and ratings of information sources for medical decisions: results from the DECISIONS survey, Med Decis Making 30: ((2010) ), 106–114. |
[16] | Chen Y.Y. , Li C.M. , Liang J.C. and Tsai C.C. , Health Information Obtained From the Internet and Changes in Medical Decision Making: Questionnaire Development and Cross-Sectional Survey, J Med Internet Res 12: ((2018) ), e47. |
[17] | D’alessandro D.M. , Kingsley P. and Johnson-West J. , The readability of pediatric patient education materials on the World Wide Web, Archives of Pediatrics & Adolescent Medicine 155: ((2001) ), 807–812. |
[18] | Edmunds M.R. , Barry R.J. and Denniston A.K. , Readability assessment of online ophthalmic patient information,” JAMA Ophthalmology 131: ((2013) ), 1610–1616. |
[19] | Edmunds M.R. , Denniston A.K. , Boelaert K. , Franklyn J.A. and Durrani O.M. , Patient information in graves’ disease and thyroid-associated ophthalmopathy: readability assessment of online resources, Thyroid 24: ((2014) ), 67–72. |
[20] | El-Shunnar S.K. , Hoare D.J. , Smith S. , Gander P.E. , Kang S. , Fackrell K. and Hall D.A. , Primary care for tinnitus: practice and opinion among GPs in England, J Eval Clin Pract 17: ((2011) ), 684–692. |
[21] | Fast M. , Deibert C.M. , Hruby G.W. and Glassberg K.I. , Evaluating the quality of Internet health resources in pediatric urology, Journal of Pediatric Urology 9: ((2013) ), 151–156. |
[22] | Flesh R.F. , A new readability yardstick, Journal of Applied Psychology 32: ((1948) ), 221–233. |
[23] | Fox S. , Americans feel better informed thanks to the Internet, 2014. Retrieved from: http://www.pewInternet.org/files/2014/12/PI InformedWeb 120814 02.pdf. [Accessed October 20, 2018]. |
[24] | Gemoets D , Rosemblat G , Tse T. and Logan R. , Assessing readability of consumer health information: an exploratory study, Stud Health Technol Inform 107: (Pt 2) ((2004) ), 869–73. |
[25] | GoogleTrends. 2018. GoogleTrends. [ONLINE], Retrieved from https://trends.google.com/trends/ [Accessed 22 October 2018]. |
[26] | Grewal P. and Alagaratnam S. , The quality and readability of colorectal cancer information on the Internet, International Journal of Surgery 11: ((2013) ), 410–413. |
[27] | Hooks R. , Patient-centred care, Nursing Standard 30: (2016), 61–62. |
[28] | Jayaweera J.M.U. and De Zoysa M.I.M. , Quality of information available over Internet on laparoscopic cholecystectomy, Journal of Minimal Access Surgery 12: ((2016) ), 321–324. |
[29] | Kirsch I.S. , Jungeblut A. , Jenkins L. , et al., Adult literacy in America: A first look at the results of the National Adult Literacy Survey, Washington, DC: U.S. Department of Education, (1993). |
[30] | Klila H. , Chatton A. , Zermatten A. , Khan R. , Preisig M. and Khazaal Y. , Quality of web-based information on obsessive compulsive disorder, Neuropsychiatric Disease and Treatment 9: ((2013) ), 1717–1723. |
[31] | Krippendorff K. , Inferring the readability of text. In: K. Krippendorff, M.A Bock, The Content Analysis Reader (2008), pp. 202–208. |
[32] | Küçükdurmaz F. , Gomez M.M. , Secrist E. and Parvizi J. , Reliability, readability and quality of online information about femoracetabular impingement, Arch Bone Jt Surg 3: ((2015) ), 163–168. |
[33] | Kutner M. , Greenberg E. , Jin Y. and Paulsen C. , The Health Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006–483). U.S. Department of Education. Washington, DC: National Center for Education Statistics, (2006). |
[34] | Laplante-Lévesque A. , Brännström K.J. , Andersson G. and Lunner T. , Quality and readability of English-language Internet information for adults with hearing impairment and their significant others, Int J Audiol 51: ((2012) ), 618–626. |
[35] | Laplante-Lévesque A. and Thoren E.S. , Readability of Internet Information on Hearing: Systematic Literature Review, American Journal of Audiology 24: ((2015) ), 284–288. |
[36] | Lee K. , Hoti K. , Hughes J.D. and Emmerton L. , Dr Google and the consumer: a qualitative study exploring the navigational needs and online health information-seeking behaviors of consumers with chronic health conditions, Journal of Medical Internet Research 16: ((2014) ), e262. |
[37] | Manchaiah V. , Dockens A.L. , Flagge A. , Bellon-Harn M. , Azios J.H. , Kelly-Campbell R.J. and Andersson G. , Quality and readability of English-Language Internet information for tinnitus, Journal of the American Academy of Audiology (2018), doi 103766/jaaa.17070 |
[38] | McKearney R.M. , MacKinnon R.C. , Smith M. and Baker R. , Tinnitus information online - does it ring true? J Laryngol Otol 24: ((2018) ), 1–6. |
[39] | Mc Laughlin G.H. , SMOG grading-a new readability formula, Journal of Reading 12: ((1969) ), 639–646. |
[40] | McNally S.L. , Donohue M.C. , Newton K.P. , Ogletree S.P. , Conner K.K. , Ingegneri S.E. and Kagnoff M.F. , Can consumers trust web-based information about celiac disease? Accuracy, comprehensiveness, transparency, and readability of information on the internet, Interact J Med Res 1: ((2012) ), e1. |
[41] | Meade C.D. and Smith C.F. , Readability formulas: cautions and criteria, Patient Education and Counseling 17: ((1991) ), 153–158. |
[42] | Memon M. , Ginsberg L. , Simunovic N. , Ristevski B. , Bhandari M. and Kleinlugtenbelt Y.V. , Quality of web-based information for the 10 most common fractures, Interact J Med Res 17: ((2017) ), e19. |
[43] | Miller L.M.S. and Bell R.A. , Online health information seeking: the influence of age, information trustworthiness, and search challenges, Journal of Aging and Health 24: ((2012) ), 525–541. |
[44] | Moody E.M. , Clemens K.K. , Storsley L. , Waterman A. , Parikh C.R. and Garg Donor A.X. , Nephrectomy, Improving on-line information for potential living kidney donors, Kidney Int 71: ((2017) ), 1062–1070. |
[45] | Murdin L. and Schilder A.G. , Epidemiology of balance symptoms and disorders in the community: a systematic review, Otology & Neurotology 36: ((2015) ), 387–392. |
[46] | National Center for Education Statistics. (2003). National Assessment of Adult Literacy. U.S. Department of Education Retrieved from http://nces.ed.gov/naal/. [Accessed on 24 October 2018]. |
[47] | Oh H.J. and Lee B. , The effect of computer-mediated social support in online communities on patient empowerment and doctor-patient communication, Health Commun 27: ((2012) ), 30–41. |
[48] | Oleander Software [Computer software] (2015), Retrieved from http://www.oleandersolutions.com. |
[49] | Pulvirenti M. , McMillan J. and Lawn S. , Empowerment, patient centred care and self-management, Health Expectations 17: ((2014) ), 303–310. |
[50] | Raj S. , Sharma V.L. , Singh A.J. and Goel S. , Evaluation of Quality and Readability of Health Information Websites Identified through India’s Major Search Engines, Adv Prev Med (2016), 4815285. |
[51] | Rew L. , Saenz A. and Walker L.O. , A systematic method for reviewing and analysing health information on consumer-oriented websites, J Adv Nurs 74: ((2018) ), 2218–2226. |
[52] | Sbaffi L. and Rowley J. , Trust and Credibility in Web-Based Health Information: A Review and Agenda for Future Research, J Med Internet Res 19: ((2017) ), e18. |
[53] | Seçkin G. , Yeatts D. , Hughes S. , Hudson C. and Bell V. , Being an informed consumer of health information and assessment of electronic health literacy in a national sample of Internet users: validity and reliability of the e-HLS instrument, Journal of medical Internet research 18: ((2016) ), 7. |
[54] | Soobrah R. and Clark S.K. , Your patient information website: how good is it? Colerectal Dis 14: ((2012) ), e90–4. |
[55] | Tulbert B.H. , Snyder C.W. and Brodell R.T. , Readability of patient-oriented online dermatology resources, J Clin Aesthet Dermatol 4: ((2011) ), 27–33. |
[56] | Walsh T.M. and Volsko T.A. , Readability assessment of Internet-based consumer health information, Respiratory Care 53: ((2018) ), 1310–1315. |
[57] | Weiss B.D. , Health Literacy: A Manual for Clinicians. Chicago, IL: American Medical Association Foundation, ((2003) ). |
[58] | Wyatt C. , Commentary: measuring quality and impact of the World Wide Web, e British Medical Journal 314: ((1997) ), 1879–1881. |
[59] | Zhang Y. , Sun Y. and Xie B. , Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results, Journal of the Association for Information Science and Technology 66: ((2015) ), 2071–2084. |
