Affiliations: [a]
Mechatronics Engineering Department, Faculty of Technology, ANS campus, Afyon Kocatepe University, Afyonkarahisar, Türkiye | [b]
The Institute of Graduate Studies, Karabuk University, Karabük, Türkiye
Abstract: This work aimed to develop a data glove for the real-time translation of Turkish sign language. In addition, a novel Fuzzy Logic Assisted ELM method (FLA-ELM) for hand gesture classification is proposed. In order to acquire motion information from the gloves, 12 flexibility sensors, two inertial sensors, and 10 Hall sensors were employed. The NVIDIA Jetson Nano, a small pocketable minicomputer, was used to run the recognition software. A total of 34 signal information was gathered from the sensors, and feature matrices were generated in the form of time series for each word. In addition, an algorithm based on Euclidean distance has been developed to detect end-points between adjacent words in a sentence. In addition to the proposed method, CNN and classical ANN methods, whose model was created by us, were used in sign language recognition experiments, and the results were compared. For each classified word, samples were collected from 25 different signers, and 3000 sample data were obtained for 120 words. Furthermore, the dataset’s size was reduced using PCA, and the results of the newly created datasets were compared to the reference results. In the performance tests, single words and three-word sentences were translated with an accuracy of up to 96.8% and a minimum 2.4 ms processing time.
Keywords: Extreme learning machines (ELM), fuzzy logic, sign language recognition, data glove, CNN, ANN
