A plea for systematic literature analysis and conclusive study design, comment on: “Systematic review of magnetic resonance imaging for diagnosis of Meniere disease”

Lopez-Escamez and Attyé present a review article on the topic of endolymphatic hydrops (ELH) imaging [40]. Within the last decade, ELH imaging has greatly impacted the field of otology and has prompted an ongoing shift in the diagnostic paradigm for Menière’s disease (MD) from a purely clinical-based diagnosis to one that also takes into account objective visualization of the chief histopathologic correlate known to occur in this condition—namely endolymphatic hydrops (MD)∼ [20, 21]. Because the traditional symptom-based systems used to diagnose MD have long been known to lack ideal specificity and sensitivity, it is therefore of great clinical interest to systematically analyze the published literature on this topic since 2007, and in this regard, we praise the authors’ intent.

Nevertheless, several aspects of this review are concerning, and we believe the main presented conclusions are overly simplified and not entirely supported by a full consideration of the relevant literature. Some of these issues likely stem from the methodology applied in this effort, but also the review does not really systematically seek to study a specifically pre-defined primary outcome measure. Rather, it presents a limited listing of articles and then provides generalized commentary on several issues related to imaging of MD. In the following, we present a few key concerns in order to offer an alternative broader view onto this topic.

One major area of concern relates to the unclear and seemingly inconsistent basis for study inclusion. First, the authors excluded studies without control groups or studies that used the contralateral ear as a control; yet, details of what constituted an acceptable control group are not provided. Second, the authors excluded studies from the years 2007–2012, without offering a clear justification for this temporal selectivity. This is concerning as clinical ELH imaging is being performed since 2007. Third, the inclusion criteria do not require defined clinical diagnostic criteria of MD for patient selection. In fact, their presented list of included studies lists 3 studies (i.e. 23% of all included studies) that did not report any MD diagnostic criteria. Forth, eight controlled studies, four from the years 2010–2012 [5, 9, 36, 39] and another four from the years 2014–2017 [8, 25, 28, 29], were not included, for reasons that are not entirely clear (Table 1). All of these studies included healthy controls and seemingly should have been included. Lastly, there is no flow diagram to outline search results, inclusion, and exclusion as is generally recommended in a systematic review according to PRISMA guidelines.

It appears difficult to pursue the authors’ main stated objective of this review (to determine if visualization of ELH on MRI can be used as diagnostic criteria for MD) while excluding a large number of controlled studies which appear to fulfill the presented inclusion criteria. Furthermore, the authors state that in order to pursue this main objective they retrieved certain data from the selected studies such as duration of disease (reported in only 2 of the selected studies) and technical MR parameters. However, they did not retrieve data fundamental to their stated aim such as the proportion of patients correctly diagnosed by MR imaging as compared to a narrowly defined clinical standard (such as define MD) or by other electrophysiologic diagnostic methods. These data would have been necessary to examine the main hypothesis of this review (can visualization of EH on MRI be used as diagnostic criteria for MD?).

In the authors’ inclusion criteria, they allowed inclusion of patients with vestibular migraine, but only according to the diagnostic criteria published in 2012 by Lempert et al. [37], and not the diagnostic criteria published by Neuhauser and Lempert in 2009 [45]. This decision seems somewhat arbitrary when considering the accepted criteria for MD originate from 1995, and also considering that the difference between the 2009 and the 2012 criteria for VM is subtle. This selection strategy, for example, led to the exclusion of the publication from the year 2013 [23], which demonstrated for the first time in the published literature the presence of endolymphatic hydrops in patients with the symptom-based diagnosis of vestibular migraine (according to the 2009 criteria by Neuhauser et al.). This discovery is of extraordinary importance for the understanding of MD and VM and the clinical picture of patients with ELH.

Furthermore, concerning the diagnostic criteria, the authors make no distinction between the clinically disparate patient groups of “definite”, “probable” and “possible” MD. This is a major problem because opening analysis to probable and possible cases would bias conclusions drawn concerning the main hypothesis of this review unless their intent had been to correlate these sub groups of MD with imaging outcomes, but that was not the case.

Under the heading “Knowledge from controlled studies”, the authors – rather than presenting the usual sort of data such as sensitivity/specificity – merely provide commentary on aspects of ELH imaging that have little to do with data gleaned from systematically reviewing the articles included.

For example, the different MR hydrops grading systems have been developed and published in non-controlled studies. The authors mention only two grading methods (the 3-stage grading by Naganawa et al. and their own 2-stage grading), but they do not mention, for example, the grading proposed by Barath et al. [1] or the 4-stage grading proposed by Bernaerts et al. [2] or the 4-stage grading [19] and volumetric grading by Gürkov et al. [13]. The latter of these 4-stage grading scales has been used for generating novel insights into the structure-function-correlations between ELH severity and audiovestibular functions [18, 19, 31] and for a longitudinal follow-up study of the effect of placebo treatment on ELH severity [16]. The volumetric grading method has established a semi-automated quantification method for cochleovestibular ELH and has been used to confirm the strong positive correlation between hearing loss and ELH severity in MD patients [13].

Once again, the stated aim of the review article was to investigate if visualization of EH on MRI can be used as diagnostic criteria for MD, but it is hard to see how that question could possibly have been answered by the data that the authors actually present. The main table (Table 2) merely lists some studies with a control group, but actually present no data that is particularly useful to accomplish the main purpose. In fact, the only instance in the entire paper where the authors present data about sensitivity and specificity is a citation from one single study [47] that did not even use any contrast agent for the visualization of ELH. This method is possibly based on artefacts [4] and furthermore questionable, since endolymph and perilymph currently cannot be reliably distinguished without any contrast enhancement. Moreover, four of the 13 included studies did not use any contrast agent, and a further two studies [6, 26] were not even concerned with ELH imaging at all.

Under the heading “differential diagnoses for MD”, the authors selectively mention one single study from their own group. Interestingly, this study found ELH in only 48% of patients with clinically definite MD, whereas a large number of studies from other groups found rates of 100% or close to 100% of ELH in patients with clinically definite MD [1, 3, 5, 18, 19, 30, 36, 39, 46, 49, 50, 52]. In other words, the authors’ own paper that is discussed is in fact an outlier within the scope of the relevant medical literature, but this discrepancy is not discussed.

The authors further state that their own referenced study showed that patients with MD-like symptoms can also have inner ear lesions such as malformations. This is not a new finding, as it has been reported previously [27, 44]. Furthermore, several other previous publications (e.g. [7, 22, 33, 34, 51] from other groups have documented ELH in patients with various inner ear lesions —a phenomenon equally well documented in histopathological temporal bone studies.

Perhaps one of the most significant concepts that has emerged from imaging of ELH is the realization that “clinical” MD likely only represents a subset of patients with ELH and clinical symptoms—who often do not fit the “typical” clinical definition of MD (reviewed in [24]), as for example vertigo may be absent. In other words, the old symptom-based concept of MD is simply too narrow to fully account for the complete range of symptomatic ELH. This has led to the development of the clinical-practical concept of Hydropic Ear Disease, which defines a larger spectrum of ELH that includes clinical variants as well as the primary and secondary forms of MD within one single classification [10, 11, 20, 21].

The authors state that they have not found any longitudinal follow-up studies. However, a cursory Medline search will reveal several longitudinal follow-up studies, both with and without control groups [8, 15, 32, 38].

Under the heading “conclusions on MRI findings in MD”, the authors state three main conclusions. The first conclusion claims that ELH could be reliably measured in the saccule using the SURI method. However, such a conclusion is not supported by the presented data. In order to examine this hypothesis, an entirely different study design of the review article would have been necessary (that would have included relevant saccular imaging data points). Also, to this purpose, it is irrelevant whether an MRI study has a control group or not, because the ability to measure an anatomic structure does not depend on the patient’s symptoms, but on many other (mostly technical) factors. These factors include the (in)ability to differentiate two adjacent structures, e.g. utriculus and sacculus with the same signal intensity and without a visible border between them. Besides this questionable SURI method, the sacculus utriculus confluence criterion has been proposed by Bernaerts et al. [2]. Related, the conclusion presented by the authors in the abstract “studies have identified the saccule as the most specifically involved structure in MD” similarly has no justification based on the work done in this study and is seemingly just a recapitulation of the authors’ previously publish edopinions.

The second main conclusion of the review article relates to the association between endolymphatic hydrops and hearing loss. While this point has previously been discovered in 2011 [19] and has been confirmed several times in the previous literature (e.g. [8, 13, 28]), the larger point remains that this review does not systematically present hearing data. To examine such a hypothesis with a systematic review another study design that would include hearing status data review would have been advisable.

The third and final conclusion of the review article states that uncontrolled studies have described ELH in patients with vestibular migraine, acute low tone SNHL (without vertigo) and in other clinical pictures. However, there are no references given to these studies (e.g. [23]).

The authors state that they “have selected case-controlled studies to better characterize the potential added value in the diagnosis”. This is a problematic concept. If one accepts MRI diagnosis as a “new test” and the symptom-based diagnosis as the “gold-standard” as seems to be implied by the authors’ assertion that MD is a clinical diagnosis, then it is impossible to characterize the “added diagnostic value” of the MRI. With such an approach, the MRI diagnosis can never be as good as or better than the simple clinical diagnosis. There would be no clinical sense in performing an expensive MRI if the best possible diagnostic result would be a confirmation of a diagnosis which is already established by a ten-minute interview and a simple audiogram. Yet, the troublesome limitations of clinical-based diagnosis are in fact the driving force that has motivated researchers to apply MRI in the diagnostic approach for patients suspected of having MD or other forms of Hydropic Ear Disease. The rationale behind MR imaging of ELH is in fact based on a desire to seek improved diagnostic precision based on the proven histopathological correlation between ELH and MD. It is an attempt to visualize clinically what is known to be present post mortem in order to work toward an improved diagnostic “gold-standard.”

In our experience, the real added clinical value of the hydrops MRI is the identification of patients who would be missed by the simple history-based clinical diagnostic criteria [21]. Therefore, to assess the potential added value of MRI, it would be advisable to examine patients with undiagnosed audiovestibular symptoms who would not be diagnosed with definite MD by using the simple clinical criteria.

The authors state that “Meniere disease (MD) is a clinical syndrome”. This opinion, however, should not be considered in isolation. Instead, alternative expert opinions should equally be considered and cited, e.g. the AAO-HNS classifications of 1972, 1985 AND 1995 as well as the new classification of Hydropic Ear Disease [20], all of which are in clear contradiction to this opinion. The authors’ opinion to regard MD as a mere “syndrome” does not appear to have a conclusive scientific basis. The pathognomonic pathological sign is well-recognized to be endolymphatic hydrops. Therefore the terms of “syndrome” and “disease” should not be confused, as it is the case in the recently proposed clinical diagnostic criteria [20]. We are considering MD as a disease and therefore the disease concept includes not only certain symptoms but also the underlying pathology, i.e. ELH.

The authors state that “clinicians should interpret these images with caution to exclude description of endolymph in non-physiologic compartment due to geometric coregistration problems between the two sequences [18].“ However, here the authors cite their own letter to the editor. In this letter, the authors make false statements about an article by another group [10] and wrongly claim that another article by this group [32] had used a coregistration method, which is a false speculation and has already been publicly disproven [12]. It seems hard to understand why the authors are now making the same wrong and disproven insinuation again in the format of a review article.

The authors state that intratympanic contrast injection “can cause local toxicity in animal models [33].” However, their review article is concerned with a clinical question (can visualization of EH on MRI be used as diagnostic criteria for MD?), and therefore it seems hard to understand why the authors only cite one single animal study in which a non-clinical Gd dilution was used. In contrast, the authors do not cite any of the clinical studies (e.g. [41–43]) that have examined the potential ototoxic effects of clinically used intratympanic Gd applications in humans. Furthermore, the clinically used Gd dilution was also found to be safe (endocochlear potential, electron microscopy) in an animal study [35].

The authors state that “A recent meta-analysis of temporal bone studies also proposed a cochleocentric distribution of the endolymph fluid with constant cochlear duct dilatation in healthy subjects [41], explaining the frequent visualization of endolymph in normal cochlear ducts with MRI.” However, this narrative is misleading. The first of the two original studies referenced therein is from Minnesota [48], from the year 1993. It included 13 bones from 11 patients. These 11 patients were chosen from the temporal bone collection based on the presence of ELH and the absence of typical MD symptoms (i.e. fluctuating hearing loss and vertigo attacks). Of these 11 patients, 8 had otopathology such as chronic otitis media, otosclerosis, serous labyrinthitis, acute otitis media with effusion, and metastatic carcinoma. In 11 of the 13 bones, the ELH was only apical. The second original study [53] was from Massachussetts, from the year 1982. It examined 495 bones from 300 patients. The majority of these patients had otopathologic diagnoses of chronic otitis media and otosclerosis. They found apical ELH in 78 bones from 66 subjects. The incidence of apical ELH for the entire population studied was 15.8%. Therefore, to consider the patients of these temporal bone studies as “healthy subjects” is misleading as they in fact had various forms of ear disease.

Furthermore, with their statement “constant cochlear duct dilation in healthy subjects” they convey the impression that ELH was something non-pathologic. However, an abnormal distension of the endolymph space, i.e. endolymphatic hydrops, is clearly established as a significant inner ear pathology, and a recent in-vivo study revealed that Hydropic Ear Disease is associated with a broad array of audiovestibular symptoms [21], and not only the simplified typical symptoms used in the clinical diagnostic criteria.

The authors state that “The absence of inclusion of grading for utricular protrusion into the lateral semicircular canal following IT contrast media injection [45] could limit this classification’s [SURI grading] usefulness.” However, this citation appears misleading, since it is referring to a letter void of original research data. The MRI discovery of hydropic herniation from the vestibulum into the semicircular canal has originally been described by Gürkov et al. in 2012 [14, 17].

In summary, this article is an apparently inconsistent execution of a systematic review that seemingly displays deficient and contradictory inclusion criteria that are also inconsistently applied. A large number of relevant studies are unfortunately omitted for reasons unknown. The data presented are not framed in a manner that allow evidence to suggest an answer to a clearly-stated hypothesis. The conclusions that are presented cannot be drawn from the actual data gleaned by systematically reviewing the literature, nor is the underlying design suitable to support most of these conclusions. The result of these deficiencies is an article that will likely confuse most readers and possibly be interpreted in a way that would partially discredit MRI as a viable emerging tool in the diagnosis of the spectrum of ear diseases. We propose that systematic reviews on ELH imaging be conducted in a transparent and conclusive manner in the future.