Detection method for unrecognized spatial disorientation based on optical flow stimuli

1.Introduction

Spatial disorientation (SD) is a pilot’s erroneous perception of position, attitude, or motion in relation to the gravitational vertical angle and the Earth’s surface. Examples include illusions of self-motion (or vection) [1] and autokinetic illusion [2]. SD has caused many flight accidents so far. From 1993 to 2013, 11% of the flight accidents in the US troops were related to SD [3], with a mortality of 69% [4]. Hence, SD poses a serious threat to flight safety [5].

SD-induced flight accidents are generally divided into three categories [6]: unrecognized SD in this category, a pilot gets disoriented but does not recognize it [7]; recognized SD is aware of SD [8]; and uncontrollable SD is aware of SD, but the pilots gets incapacitated and loses control of the aircraft [7]. According to the latest research, the three types of SD correspond to the three stages of human brain cognition. The three of them are interconnected, each building on the previous one [9]. The judgment method varies for each type of SD. Recognized SD and incontrollable SD can be more intuitively determined, for example, by subjective judgment [10] and rating scales [11]. However, the judgment method for unrecognized SD is not yet mature. In this study, we focused on the detection method for unrecognized SD to solve these mentioned problems.

Although unrecognized SD goes undetected by the human, it influences human behaviors subconsciously. From 2002 to the present, the changes in the center of foot pressure (CoP) were detected using a force plate in several studies on vision-induced vection illusion and postural deviation [12]. CoP deviation was found to be larger during self-reported vection illusion than in its absence.

In 2002, Thurrell et al. [13] examined whether the conscious awareness of self-motion affected the amplitude and directional accuracy of vision-induced postural response. Among studies on vision-induced vection illusion and postural deviation, the changes in CoP were detected using a force plate. CoP deviation was larger during self-reported vection illusion than in its absence. The results confirmed that the direction of vision-induced postural response changed with the variation in the visual stimuli varied. In 2005, Fushiki et al. [14] conducted a similar study and used video-based motion analysis equipment to discuss the subjects’ head and trunk deviations during the experiment. They concluded that postural balance was affected by vection illusion.

Da-Silva et al. [15] conducted an experimental study in 2015 using dynamic virtual vision simulation to induce vection. The CoP data in different directions and under different speeds of the visual scene were analyzed. Their study showed that the displacement of CoP upon vection had the same direction as the visual scene. Their study also showed that the postural balance was disrupted by vection, resulting in significant changes in CoP.

The aforementioned studies were concerned with postural balance upon vection. However, very few focused on recognized SD. Mounting evidence shows that the differences in postural instability can be used to predict who will experience strong illusory self-motions (vection) and become sick when exposed to global patterns of optical flow [16, 17]. Observing a moving visual scene is very likely to generate a feeling of disorientation [18], which inevitably results in postural adjustments [19]. When SD is induced by only visual information, very small discrepancies are noted in posture and individuals’ susceptibility to SD. The subjects may have little awareness of the occurrence of SD. This situation is known as unrecognized SD, which poses the greatest threat to flight safety. Even if the pilots themselves are unaware of SD, we can still detect it from minor postural swings. On this basis, we could determine the occurrence of SD and differentiate between different types of SD events, especially unrecognized SD. Then, enhancing SD-targeted training based on this understanding is an essential means of reducing flight accidents.

In the present study, we applied a judgment methodology for unrecognized SD published in 2020 [20] and 2022 [9] to practice. CoP was used as an indicator to determine the occurrence of unrecognized SD. CoP was combined with subjective judgment to determine the occurrence of unrecognized SD. The present study explored the feasibility of the judgment methodology in two different populations: pilots and non-pilots. We investigated the feasibility and reliability of this methodology and assessed the similarities and differences in the experimental induction of unrecognized SD in pilots and non-pilots. Therefore, we hope that this study can provide reference for reducing flight accidents and casualties.

2.Method

2.3Visual stimuli

The optical flow stimuli were given to induce SD. The video was shown on a 2 × 2 m² projection screen. Only two colors were displayed, that is, black (R: 255, G: 255, and B: 255) and white (R:1, G:1, and B:1). A visual stimulus was projected on the screen [21] to simulate the scene in which the light observable on the ground from a high-speed aircraft at night was moving toward the pilot due to relative motion.

Figure 1.

Optical flow stimuli.

3.Results

3.2Postural swing analysis

We first calculated the S of CoP for each participant in the control tests. The upper bound of the 95% confidence interval was calculated using the standard deviation of the six control tests for each participant. These upper bounds within each group were arranged in ascending order, as shown in Table 2 and Fig. 2. Then, the t test (α= 0.05) was used to compare the confidence intervals of the mean. The t test showed significant differences in the means between pilots and non-pilots (P= 0.03). This result indicated a significantly lower upper bound of the confidence interval in pilots than in non-pilots. The physically stationary pilots better controlled standing balance despite potential vection, indicating a better spatial orientation ability than in non-pilots.

Table 3

S values of each test in participants from the two groups. The values marked by ^∗ fell on the right side of the confidence interval

No.	Pilot group						Non-pilot group
	Low-speed tests			High-speed tests			Low-speed tests			High-speed tests
	1	2	3	1	2	3	1	2	3	1	2	3
1	0.17	0.14	0.11	0.14	0.15	0.10	0.36	0.40*	0.37	0.43*	0.39*	0.47*
2	0.20	0.24	0.27	0.30*	0.33*	0.19	0.74*	0.40	0.55*	0.38	0.50*	0.56*
3	0.26	0.20	0.24	0.44*	0.27	0.25	0.19	0.17	0.78*	0.20*	0.26*	0.24*
4	0.29*	0.24	0.19	0.26	0.32*	0.36*	0.15	0.24*	0.26*	0.24*	0.20	0.17
5	0.35	0.51*	0.40*	0.29	0.33	0.31	0.57*	0.59*	0.51*	0.43	0.41	0.39
6	0.49	0.33	0.48	0.45	0.47	0.82*	0.82*	0.54	0.77*	0.58*	0.63*	0.71*
7	0.25*	0.26*	0.28*	0.21	0.24	0.19	0.38	0.30	0.40	0.43*	0.51*	0.39
8	0.38*	0.28*	0.23	0.31*	0.25	0.40*	0.13	0.17	0.15	0.12	0.10	0.16
9	0.39	0.49*	0.39	0.55*	0.40	0.33	0.52*	0.43	0.40	0.61*	0.41	0.57*
10	0.40	0.36	0.30	0.48*	0.40	0.37	0.80*	0.37	0.42	0.86*	0.49	0.76*

Figure 2.

Upper bounds of the 95% confidence interval in the two groups receiving the control tests.

After obtaining the upper bounds of the 95% confidence interval, we further calculated the standard deviation of CoP for each participant in each test (Table 3). If S for a specific participant in a test was above the upper bound of the confidence interval, the S of CoP of this participant in this test fell on the right side of the confidence interval. In that case, the swing amplitude would be much more significant than that in the control group. Such postural imbalance was caused by SD. Table 3 shows the values of S of CoP for each participant in each test in the pilot and non-pilot groups, respectively. The values marked by ^∗ fell on the right side of the confidence interval.

As shown in Table 3, in low-speed optical flow stimuli tests from pilots, the S values fell on the right side of the confidence interval in nine tests. Postural imbalance events occurred in these participants, accounting for one third of all tests. In high-speed optical flow stimuli tests from pilots, S values fell on the right side of the confidence interval in 10 tests. Postural imbalance events occurred in these participants, accounting for one third of all tests. In low-speed optical flow stimuli tests from non-pilots, S values fell on the right side of the confidence interval in 13 tests. Postural imbalance events occurred in these participants, accounting for two thirds of all tests. In high-speed optical flow stimuli tests from non-pilots, S values fell on the right side of the confidence interval in 18 tests. Postural imbalance events occurred in these participants, accounting for one-half of all tests.

4.Discussion

We administered video-based optical flow stimuli to 10 pilots and 10 non-pilots for the experimental induction of SD. A set of criteria were set up for identifying unrecognized SD. The purpose was to discuss the feasibility of the judgment method for unrecognized SD and the difference between pilots and non-pilots in terms of susceptibility to SD.

The analysis of subjective judgment data from the participants showed that, under the visual stimuli at the same flow speed, vection was reported less frequently in pilots than in non-pilots. One possible reason was that the pilots received flight simulator training addressing the vection problem as part of their routine training. In contrast, non-pilots were rarely exposed to such scenes and were more likely to experience recognized vection compared with pilots. The aforementioned finding proved the effectiveness of flight simulator training in reducing SD [23, 24]. Some researchers believed that the three types of SD events were three successive steps in human perception. Increasing the level of cognition could help with the timely recognition of SD [20].

The present study demonstrated the feasibility of the judgment method for unrecognized SD. The subjective judgment data were combined with the statistical analysis of CoP to identify participants and tests positive for unrecognized SD. Twenty participants recruited in the present study received 120 control tests in total. The statistical analysis of the confidence intervals calculated from the standard deviation (S) of CoP in the control tests in the two groups showed that the pilots better controlled their postural balance when being physically stationary in a standing position compared with non-pilots. In the tests, S values of CoP falling on the right side of the confidence interval indicated that the postural balance was disrupted. In other words, the participants could not maintain postural balance in a specific test. This finding agreed with those of PDa-Silva et al. [15] in 2015: if the postural balance was disrupted by vection, CoP would change considerably.

We used the proposed judgment method for unrecognized SD. The unrecognized SD events totaled 43 in 20 participants, accounting for 35.8% of all tests. This result agreed with the general occurrence rule of unrecognized SD [25]. The total number of unrecognized SD events occurring in pilots was reduced by 18.3% compared with that in non-pilots. Although the difference was not statistically significant, we have reasons to believe that pilots are less likely to experience SD under optical visual flow stimuli, probably due to frequent training addressing SD. It was thus proved feasible to combine subjective and objective judgments to determine the occurrence of recognized SD. Our study shed new light on the detection of unrecognized SD [26] and laid the basis for subsequent experimental induction and statistical analysis of SD. Hence, enhancing SD-targeted training may be an effective pathway to reduce SD-related accidents.

Two-way ANOVA, considering the interaction effect between variables based on all test data, indicated no significant difference in SD occurrence under different population and flow speed factors. This result was inconsistent with the findings of Da Silva et al. [15] . The latter believed that the flow direction of the visual scene affected people’s control of postural balance. The discrepancy might be explained by the slight difference between the two flow speeds adopted for our tests. Another probable reason was the limited number of participants recruited for the present study, leading to an insignificant difference. We will follow up on this issue in the future.

To conclude, we identified participants and tests positive for unrecognized SD by combining subjective judgment with the standard deviations (S) of CoP. Under optical flow stimuli, the number of recognized and unrecognized SD events in pilots was smaller than that in non-pilots. Besides, the pilots had better control of postural balance in the control tests and the optical flow stimuli tests than the non-pilots. These results demonstrated that weak stimuli or visual stimuli alone resulted in a very small amplitude of postural imbalance. The participants themselves were hardly aware of the occurrence of SD. This situation is known as unrecognized SD [27], which poses the greatest threat to flight safety. Even if the pilots themselves are unaware of SD, we can still detect them from minor postural swings. Combining subjective judgment with postural swing analysis proved effective for identifying unrecognized SD [26]. Besides, our results agreed with the findings of Nam-Gyoon Kim and Beom-Su Kim [28], confirming the assumption that severe SD was associated with worse control of postural balance.

The aforementioned findings suggested that the amplitude of postural deviation was closely related to the occurrence of SD. Many studies have shown that the involuntary deviation of the head or trunk occurs first upon SD, resulting in CoP deviation. In future research, we will use this method to analyze more subjects. We plan to combine this judgment method with the EEG signals. Used to analyze whether there are observable characteristic changes in EEG signals when unrecognized SD occurs, and further improve the judgment method for unrecognized SD.

5.Conclusion

Our research methodology opened a new direction for the study of unrecognized SD. The test results showed that combining subjective judgment with CoP data was feasible for identifying unrecognized SD. Based on this method, we will look for the characteristic changes in objective physiological signals accompanying the occurrence of unrecognized SD. Our method laid a basis for identifying unrecognized SD, screening for pilots below the cognitive threshold, and enhancing the training addressing unrecognized SD. Our study might contribute to the reduction of SD-related flight accidents.