Exploration of quality criteria for the detection of peak inspiratory flow under different resistance conditions

1.Introduction

Chronic airway diseases affect a large segment of the population and can impose a heavy burden of disease, thus posing a major threat to global public health [1, 2]. The treatment of these diseases centers around inhalation therapy approaches in which specific medicines are delivered to the lungs through inhaler devices, including pressurized metered dose inhalers (pMDIs), soft mist inhalers (SMIs), and dry powder inhalation devices (DPIs).

DPI resistance can be separated into five levels (R1–R5, from low to high) based on the amount of inhalation flow necessary to produce a pressure drop of 4 kPa [3, 4, 5, 6]. This value was selected for use when conducting the in vitro characterization of a drug dose delivered by a DPI. When using a DPI, patients must inhale forcefully to facilitate optimal inspiratory flow and drug release [7]. If this flow rate is too low or too high, however, it will result in suboptimal pulmonary drug deposition [8, 9, 10]. Owing to airflow limitations or impaired respiratory muscle strength, individuals suffering from chronic airway diseases often face difficulties reaching this minimum flow rate [11, 12, 13]. As such, peak inspiratory flow (PIF) is an important metric that can influence the therapeutic effects of inspiration in patients undergoing inhalation therapy [14, 15].

Measuring PIF under additional resistance can provide insight into the ability of patients to use DPIs [16], while also guiding inhalation device selection and inhaler technique training [17, 18]. The number of instruments capable of measuring PIF under different levels of resistance is gradually rising. The In-Check DIAL® is a mechanical PIF meter with additional resistance. Resistance levels are divided into the R1–R5 resistance gears, which can be switched with a resistance turntable [5]. This device has been used in several studies to measure PIF values in patients, recording either the average or the highest value from three tests [19, 20, 21]. The accuracy of this instrument has not been heavily scrutinized, and Barnes et al. [22] first proposed repeatability standards for In-Check DIAL® measurement results to ensure instrument stability. Owing to growing clinical need, intelligent inhalation assessment devices continue to be developed, highlighting a need to define approaches to evaluating the performance of these instruments.

Standard flow-volume simulations or similar standardized instruments need to be used to test the performance of spirometers and other devices that measure flow rates in clinical practice to ensure that only devices meeting defined technical and quality standards are used by patients. These devices must also undergo regular testing to ensure stability and accuracy [23]. Based on the current quality testing scheme without an inhalation assessment device, the present study was conducted using a standard flow-volume simulator for quality control analyses of an inhalation assessment device with different simulated resistance levels in order to propose a quality testing method and associated standard for this device type.

2.Materials and methods

2.2Methods

Prior to testing, the devices should be kept at room temperature of 15∼25∘C normal pressure and relative humidity the simulator and devices undergoing measurement were started and allowed to preheat for 15 min (starting the devices to achieve stable performance) after which all equipment was placed in the above environment for at least 30 min. The measurement device was then connected to the interface of the standard flow-volume simulator, with the connection being air-tight (Fig. 1). The inhalation volume was set to 3.0 L using the computer, and the inhalation flow rate was set from 0–120 L/min at 15 L/min intervals (15, 30, 45, 60, 75, 90, 105, 120 L/min). After the parameters had been set, the inhalation flow rate was tested 5 times under identical conditions and the average value was recorded. The air source value generated by the simulator was used as the standard value.

Figure 1.

Test equipment operational diagram. Note: ⟀ Standard flow-volume simulator, ⟁ In-Check DIAL® (Device I), ⟂ intelligent inhalation assessment device (Device P), ⟃ computer, and ⟄ computer control terminal for Device P.

3.Results

When assessing repeatability, both Device I and Device P yielded test results that were below ± 3 L/min, consistent with the criterion for good repeatability.

The difference between test results and standard simulator values for Device P was ⩽± 5 L/min at resistance level R1 but > 5 L/min at resistance levels R2-5, whereas differences between test results and standard simulator values for Device I were greater than 5 L/min at all resistance levels. The relative error for Device P was <± 10% at resistance levels R1, R2, and R4, but >± 10% at resistance levels R3 and R5. The relative error values of Device P for velocity measurements below 1.25 L/s were all <± 10%. The relative error values for Device I at all five resistance levels were >± 10%.

Device P passed the linearity test at the R2 resistance level and partially passed the test at the R1 and R3–R5 resistance levels, while Device I partially passed the linearity test at all five resistance levels. Repeatability, absolute error, relative error, and linearity results are summarized in Tables 1 and 2.

The impedance for these two devices under a fixed pressure (4 kPa) was assessed based on the flow rate at these 5 tested resistance gear levels. Device P exhibited greater impedance than Device I at the R2 and R5 resistance levels, whereas the opposite was true at the R1, R3, and R4 resistance levels (Table 3).

In Bland-Altman analyses, most (94.50% and 96.50%) measurement results and standard simulator values at the R1-R5 levels fell within the 95% limits of agreement (LOAs), indicating that the measurements produced by these two devices were in good agreement with the standard simulator values (Fig. 2).

Figure 2.

Bland-Altman plots for measured values from these two devices and standard simulator values, with the limits of agreement extending from -1.96SD to +1.96SD (solid lines). Note: The meaning of the symbol of the test point in the chart is that the F represents Flow, the number after it represents the flow value set by the simulator, the IV and V represent the resistance gear, and the Arabic number after it is the number of tests. (A) Bland-Altman plots for measured values from Device P and standard simulator values. 94.50% (189/200) of data points were within the 95% limits of agreement (95% LOA). (B) Bland-Altman plots for measured values from Device I and standard simulator values. 96.50% (193/200) of data points were within the 95% LOA.

4.Discussion

While PIF is frequently measured with a standard spirometer, these measurements are made in the absence of any additional resistance. Some reports have demonstrated that these values are not strongly correlated with the PIF values measured with an In-Check DIAL® device [25, 26], or that the two are not correlated at all [27]. As such, PIF measures from routine pulmonary function testing cannot be used to assess inspiratory ability under conditions of resistance. The ability to assess PIF in patients exhibiting additional resistance is of great clinical significance as an auxiliary means of inhaler device selection and inhalation pattern optimization [28].

It is important that appropriate quality control testing standards be established for measurement instruments to guarantee a strong scientific basis for measurement data reliability. Spirometers, which are the devices that are most frequently used to measure inhalation and exhalation, are subject to specific quality control testing methods and standards regulated by professional institutions [23]. Similarly, peak expiratory flow meters (PEFMs) are subject to a set of mature industry standards [24, 29]. However, methods and quality assessment criteria for PIF meters under additional resistance as discussed in this paper have yet to be firmly established. Barnes et al. [22] proposed that PIF repeatability should be < 10 L/min when using the R2 (Diskus) resistance level during testing and < 5 L/min at the R5 level (Handihaler). Their results were established in patients suffering from severe chronic obstructive pulmonary disease (COPD), and repeatability in these patients may be influenced by factors including inadequate respiratory muscle strength and airway obstruction. The In-Check DIAL® has a minimum scale of 5 L/min, potentially impacting the accuracy of data analyses in the present study. The use of a standard flow-volume simulator can eliminate any variability resulting from individual patient-to-patient differences, and as such, repeatability criteria in this setting should be more strict. In the present study, both tested devices met a repeatability standard of <± 3 L/min. We thus posit that the repeatability of PIF measurements should be within ± 3 L/min under conditions of additional resistance.

Per the industry standard for PEFMs [29], the maximum allowable error for PEF measurements is within ± 10 L/min or ± 10%, owing to the large measurement values for PEF and the large degree of fluctuation in measurement error values in the absence of resistance. Under the same testing conditions in the present study, the maximum difference values measured between the standard simulator value and Device P and Device I were 0.533 L/s (≈ 32 L/min) and 0.737 L/s (≈ 44 L/min), respectively, which were measured in R5 at 2 L/s. Error increases as resistance and flow rate values increase, and as such, establishing the most appropriate error standards in a one-size-fits-all manner may not be feasible. Instead, these standards should be based upon the resistance grade. Under the same testing parameters, the maximum relative error values for Device P and Device I relative to the standard simulator value were -26.821% and -37.104%, respectively, with these values both having been recorded at resistance grade R5 at a flow rate of 2 L/s. When the flow rate was below 1.25 L/s, the relative error for Device P remained under 10%. Standard flow-volume simulators use regulated pressure to generate a specified flow rate over a range of resistance, but the target flow rate cannot be generated when the resistance level rises too high, pressure regulation fails, or other issues arise. As such, the reliability of quality control measurements made at a lower flow rate is likely to be higher. A rate of 60 L/min [30] is considered to be the node for dividing the optimal inspiratory flow rate of a DPI, and it is thus very important that measurements be reliable at flow rates proximal to 60 L/min. Here, Device P exhibited repeatability, absolute error, relative error, and linearity that met with established standard criteria at a flow rate below 60 L/min. Whether this suggests that quality testing should be conducted at moderate flow rates such as 60 L/min will require further study. In the absence of reliable quality control standards, such information will be vital to evaluate device performance in clinical practice.

Per the quality control standards for PEFMs, the linearity of a measurement instrument should be < 5% [24, 29]. Using this standard level, the devices tested herein partially based the linearity test at resistance levels R1–R5 such that these devices did not exhibit linear increasing trends with increases in analytical parameters. PEFM quality testing standards are based on measurements made in the absence of any additional resistance. The applicability of this standard to PIF meters with additional resistance will thus need to be studied further. Relative to the repeatability standard proposed by Barnes et al. [22] the present study also provides potential evaluation indices including absolute error, relative error, linearity, and impedance that may be conducive to the more rigorous and objective evaluation of device performance.

The present findings support a recommendation that a standard flow-volume simulator be used to assess the quality of PIF meters under additional levels of resistance in order to guarantee appropriate measurement accuracy. When conducting such testing, all instruments should be allowed to rest in the same environment for at least 30 min before testing to ensure that any measurement errors resulting from differences in environmental conditions are minimized. All instruments were preheated for more than 15 min before the experiment, and the connections between the tested devices and the simulator were checked to ensure tightness. Under matching parameters, each device was then measured 5 times to generate one set of data. The standard of repeatability should be <± 3 L/min for each of these datasets, while the absolute error, relative error should be ⩽± 5 L/min or <± 10%, and linearity should be < 5%, respectively (the flow rate below 60 L/min should meet the above standards).

While the measurement results for these two tested devices did not fully meet with the quality control standard proposed herein, these results and standards nonetheless offer significance as a reference. One of the reasons why these devices fail to meet established performance requirements may be that standard flow/volume simulators also have certain errors in generating standard air flow (the error of flow ⩽± 3% or ± 3L/min, error of repeatability ⩽± 2% or 3 L/min (0.05 L/s), choosing the larger value, error of linearity ⩽± 2%). To date, few reports have discussed quality control methods and standards for PIF measurements made under additional resistance, potentially owing to the type of instrument in question and the lack of any associated unified parameters. The instruments that are used for such testing are very demanding and not available to many institutions, limiting the implementation of standard quality control methodologies. Perhaps we should try more different experimental schemes to compare the evaluation effects of them, so as to explore the optimal detection methods and performance criteria. As such, additional innovative research will be vital to better clarify the optimal quality control methods and standards for these types of instruments.

5.Conclusion

A standard flow-volume simulator was used to test the quality of flow measurement instruments under additional resistance in an objective, scientifically rigorous manner. This approach to evaluate these instruments is highly conducive to further clinical application and to the more widespread use of these types of instruments.

Funding

The study was supported by the ZhongNanshan Medical Foundation of Guangdong Province (ZNSA-2020013); Key Technology R&D Program of Zhejiang, China (2020C03062); Science and Technology Innovation Project of Guangzhou Medical University in 2022–2023 (2021A001); National Key Technology R&D Program (Nos. 2018YFC1311901 and 2016YFC1304603) and National Science & Technology Pillar Program (No. 2015BAI12B10).

Author contributions

YP, ZW and KL wrote the manuscript; RH, JH, JL and SC collected and analyzed the data; YG and JZ corrected the manuscript. All authors reviewed and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Availability of data and material

All data generated or analyzed during this study are included in this published article.

Competing interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.