Industrial Applications of Genetic Algorithms:Tuning Bama Optimized Recurrent Neural Networks Using Genetic Algorithms

Previous | Table of Contents | Next

## Chapter 11
Tuning Bama Optimized Recurrent Neural Networks Using Genetic Algorithms

K. Nishita

Graduate Student
Department of Aerospace Engineering and Mechanics
University of Alabama
Box 870280
Tuscaloosa, AL 35487-0280
e-mail: knishita@eng.ua.edu

ABSTRACT

Genetic Algorithms (GAs) have demonstrated highly efficient and robust in optimization capabilities in various fields. This chapter discusses the use of a simple GA for enhancing the performance of a Bama Optimized Recurrent Neural Network (BORN). In order to measure the performance of BORN with a GA, two different studies are considered. First, the results produced by the GA optimized BORN were compared to results produced using the BORN algorithm alone. Second, a GA was used to alter the connectivities in a BORN adaptive unit to enhance the performance of the BORN algorithm. The advantage of this method is that a GA can improve the performance of the BORN by changing only the NN connections. In other words, since previously trained BORN weights are not changed, the BORN is capable of retaining its memory. Results show that a GA enhanced BORN’s adaptive capabilities with a high degree of accuracy.

INTRODUCTION

Background of GA Application in NN

Genetic Algorithms (GA) have been demonstrated highly efficient and robust optimization capabilities in various fields. In recent years, many researchers have sought to apply GAs in the area of neural networks (NNs). NNs have been successfully adapted to the number of practical engineering problems such as controllers, classifiers, etc. NNs have great potential in representing arbitrary surfaces and can be easily implemented. Various schemes for combining GA and NN have been proposed and tested to optimize NN performance in recent years. Montana and Davis [1] reported the first successful results of a relatively large NN (500 connections) optimization by GA. However, the GA used by Montana and Davis was different in several ways from the more traditional GA described by Goldberg [2]. Belew [3] used a GA to select initial NN weights and trained NN by using a gradient descent method. His results showed GA and NN combined method performed better than instances in which only a GA or a NN gradient decent method were used. Anderson [4] applied a GA for NN reinforcement learning problems in which desired outputs are not available for training purposes. He used his system to successfully control an inverted pendulum. Anderson also compared these results with his adaptive critic controller and found that GA for NN reinforcement learning was competitive with NN training method.

Recently, the most considerable attention has been focused on recurrent neural networks. The effectiveness of recurrent neural networks for the identification and control of nonlinear dynamic systems has been demonstrated by Werbos [6] and Narendra [7] et al. Wieland [5] also used GA to optimize recurrent NN for two poles jointed inverted pendulum. He used a simple GA [2] to optimize a recurrent NN controller to successfully control this difficult, highly nonlinear problem. For nonlinear system controllers such as an inverted pendulum, the effectiveness of the control system must be gauged not only by the speed of which the goal is achieved, but also by the stability and the robustness. KrishnaKumar [8,9,10] et al. demonstrated recurrent neural network stability and robustness by using the method of network connection optimization (Bama Optimized Recurrent Neural Network). Bama Optimized Recurrent Neural Network (BORN) is a back-propagation neural network (BPNN) which has the capabilities of network connectivities optimization. This optimization algorithm was first introduced by KrishnaKumar [8] for static neural networks and adapted for recurrent neural networks in [9]. BORN has demonstrated high robustness and generalization capability in various system control areas [10]. However, a disadvantage of this algorithm is that once the BORN algorithm is performed, connectivities cannot be altered without re-initializing its weights and connections. This chapter explores control capabilities of the BORN algorithm and considers a mechanism to address this disadvantage by using a GA.

Study Objective

The objective of the work presented in this chapter is to employ a GA in a BORN algorithm to alter the NN connections after the BORN algorithm execution. In order to fulfill this objective, the following studies will be undertaken:

  • Perform BPNN weight selections and connectivities optimizations using a GA and compare the results with classic BORN algorithm results.
  • Once the BORN algorithm is executed, the network connectivities cannot be adapted in BORN. In order to enhance a BORN, a GA will be used as a BORN connectivities adaptive unit for further optimal weight and connection selections. The GA adaptive unit will be executed only when BORN detects the failure to achieve its goals due to sudden changes in input patterns.

Chapter Outline

The reminder of this chapter is organized into four sections. First, BORN algorithms are introduced and the details of the mathematical model for a BORN are emphasized. Second, the details of BP Recurrent Neural Network weights selections and connectivities optimizations by GA are presented. The results of the GA optimization performance will be presented at the end of this section. Third, the concepts of GA-BORN connectivities adaptive unit are presented. Finally, the GA-BORN results are compared to those of a non-adaptive BORN.

SIMPLE GA AND BORN ALGORITHM

This section introduces simple GA and BORN algorithm. For the sake of completeness, the conventional backpropagation recurrent neural network algorithm is also presented here.

Previous | Table of Contents | Next

Copyright © CRC Press LLC