Fourier neural networks: A comparative study

Article type: Research Article

Authors: Uteuliyeva, Malika^a | Zhumekenov, Abylay^a | Takhanov, Rustem^a | Assylbekov, Zhenisbek^{a; *} | Castro, Alejandro J.^a | Kabdolov, Olzhas^b

Affiliations: [a] School of Sciences and Humanities, Nazarbayev University, Nur-Sultan, Kazakhstan | [b] Video Intelligence Lab, Huawei, Moscow, Russian Federation

Correspondence: [*] Correponding author: Zhenisbek Assylbekov, 53 Kabanbay Batyr ave., Nur-Sultan 010000, Kazakhstan. Tel.: +77 172706619; E-mail: [email protected].

Abstract: We review neural network architectures which were motivated by Fourier series and integrals and which are referred to as Fourier neural networks. These networks are empirically evaluated in synthetic and real-world tasks. Neither of them outperforms the standard neural network with sigmoid activation function in the real-world tasks. All neural networks, both Fourier and the standard one, empirically demonstrate lower approximation error than the truncated Fourier series when it comes to approximation of a known function of multiple variables.

Keywords: Neural networks, Fourier series, function approximation, convergence

DOI: 10.3233/IDA-195050

Journal: Intelligent Data Analysis, vol. 24, no. 5, pp. 1107-1120, 2020