A modified conjugate gradient-based Elman neural network |
| |
Institution: | 1. College of Information and Control Engineering, China University of Petroleum, Qingdao 266580, China;2. College of Science, China University of Petroleum, Qingdao 266580, China |
| |
Abstract: | Elman recurrent network is a representative model with feedback mechanism. Although gradient descent method has been widely used to train Elman network, it frequently leads to slow convergence. According to optimization theory, conjugate gradient method is an alternative strategy in searching the descent direction during training. In this paper, an efficient conjugate gradient method has been presented to reach the optimal solution in two ways: (1) constructing a more effective conjugate coefficient, (2) determining adaptive learning rates in terms of the generalized Armijo search method. Experiments show that the performance of the new algorithm is superior to traditional algorithms, such as gradient descent method and conjugate gradient method. In particular, the new algorithm has better performance than the evolutionary algorithm. Finally, we prove the weak and strong convergence of the presented algorithm, i.e., the gradient norm of the error function with respect to the weight vectors converges to zero and the weight sequence approaches a fixed optimal point. |
| |
Keywords: | Elman Conjugate gradient Armijo Convergence |
本文献已被 ScienceDirect 等数据库收录! |
|