Neural networks, fuzzy logic. and genetic algorithms: systhesis and applications S. Rajasekaran and G. A. Vijayalakshmi Pai

1 INTRODUCTION TO ARTIFICIAL INTELLIGENCE SYSTEMS
1.1 Neural Networks 2
1.2 Fuzzy Logic 3
1.3 Genetic Algorithms 4
1.4 Structiire of This book 5
Summary 6
References 6
Part 1 NEURAL NETWORKS
2 FUNDAMENTALS OF NEURAL NETWORKS
2.1 Basic Concepts of Neural Networks II
2.2 Human Brain 12
2.3 Model of an Artificial Neuron 13
2.4 Neural Network Architectures 16
2.4.1 Single Layer Feedforward Network 17
2.4.2 Multilayer Feedforward Network 18
2.4.3 Recurrent Networks 18
2.5 Characteristics of Neural Networks 19
2.6 Learning Methods 19
2.7 Taxonomy of Neural Network Architectures 21
2.8 History of Neural Network Research 22
2.9 Early Neural Network Architectures 23
2.9.1 Rosenblatt's Perceptron 23
2.9.2 ADALINE Network 28
2.9.3 MADALINE Network 28
2.10 Some Application Domains 30
Summary 31
Programming Assignment 31
Suggested Further Reading 32
References 32
3 BACKFROPAGATION NETWORKS
3.1 Architecture of a Backpropagation Network 35
3.1.1 The Perceptron Model 35
3.1.2 The Solution 36
3.1.3 Single Layer Artificial Neural Network 39
3.1.4 Model for Multilayer Perceptron 41
3.2 Backpropagation Learning 42
3.2.1 Input Layer Computation 43
3.2.2 Hidden Layer Computation 43
3.2.3 Output Layer Computation 44
3.2.4 Calculation of Error 45
3.2.5 Training of Neural Network 45
3.2.6 Method of Steepest Descent 47
3.2.7 Effect of Learning Rate *7]' 51
3.2.8 Adding a Momentum Term 52
3.2.9 Backpropagation Algorithm 53
3.3 Illustration 56
3.4 Applications 59
3.4.1 Design of Journal Bearing 59
3.4.2 Classification of Soil 64
3.4.3 Hot Extrusion of Steel 65 .
3.5 Effect of Tuning Parameters of the Backpropagation Neural Network
3.6 Selection of Various Parameters in BPN 72
3.6.1 Number of Hidden Nodes 72
3.6.2 Momentum Coefficient a 73
3.6.3 Sigmoidal Gain A 74
3.6.4 Local Minima 74
3.6.5 Learning Coefficient T] 74
3.7 Variations of Standard Backpropagation Algorithm 75
3.7.1 Decremental Iteration Procedure 75
3.7.2 Adaptive Backpropagation (Accelerated Learning) 7o
3.7.3 Genetic Algorithm Based Backpropagation 77
3.7.4 Quick Prop Training 77
3.7.5 Augmented BP Networks 77 , xt i
3.7.6 Sequential Learning Approach for Single Hidden Layer Neura
Networks 80
3.8 Research Directions 80
3.8.1 New Topologies 80
3.8.2 Better Learning Algorithms 81
3.8.3 Better Training Strategies 81
3.8.4 Hardware Implementation 81
3.8.5 Conscious Networks 81
Summary 82
Programming Assignment 82
References 85
associative memory
41 Autocorrelators
42 Heterocorrelators: Kosko's Discrete BAM 91
4.3
42 1 Addition and Deletion of Pattern Pairs 91
4 2 1 . —
422 Energy Function for BAM 92
Wang et al.'s Multiple Training Encoding Strategy 95
4.4 Exponential BAM 99
4.4.1 Evolution Equations 99
4.5 Associative Memory for Real-coded Pattern Pairs 101
4.5.1 Input Normalization 101
4.5.2 Evolution Equations 101
4.6 Applications 105
4.6.1 Recognition of Characters 105
4.6.2 Fabric Defect Identification 107
4.7 Recent Trends 110
Summary 111
Programming Assignment 111
References 114
ADAPTIVE RESONANCE THEORY
5.1 Introduction 117
5.1.1 Cluster Structure 117
5.1.2 Vector Quantization 118
5.1.3 Classical ART Networks 125
5.1.4' Simplified ART Architecture 126
5.2 ARTl 127
5.2.1 Architecture of ARTl 128
5.2.2 Special Features of ARTl Models 129
5.2.3 ARTl Algorithm 133
5.2.4 Illustration 135
5.3 ART2 140
5.3.1 Architecture of ART2 140
5.3.2 ART2 Algorithm 141
5.3.3 Illustration 144
5.4 Applications 145
5.4.1 Character Recognition Using ARTl 145
5.4.2 Classification of Soil 146
5.4.3 Prediction of Load from Yield Line Patterns of Elastic-Plastic
Clamped Square Plate 147
5.4.4 Chinese Character Recognition—Some Remarks 151
5.5 Sensitivities of Ordering of Data 151
Summary 152
Programming Assignment 153
Suggested Further Reading 153
References 153
FUZZY SET THEORY
6.1 Fuzzy versus Crisp 157
6.2 Crisp sets 159
Part 2 FUZZY LOGIC
6.2.1 Operations on Crisp Sets 161
6.2.2 Properties of Crisp Sets 163
6.2.3 Partition and Covering 166
6.3 Fuzzy Sets 168
6.3.1 Membership Function 169
6.3.2 Basic Fuzzy Set Operations 171
6.3.3 Properties of Fuzzy Sets 176
6.4 Crisp Relations 179
6.4.1 Cartesian Product 179
6.4.2 Other Crisp Relations 180
6.4.3 Operations on Relations 180
6.5 Fuzzy Relations 182
6.5.1 Fuzzy Cartesian Product 182
6.5.2 Operations on Fuzzy Relations 183
Summary 185
Programming Assignment 186
Suggested Further Reading 186
Reference 186
7 FUZZY SYSTEMS
7.1 Crisp Logic 187
7.1.1 Laws of Propositional Logic 189
7.1.2 Inference in Propositional Logic 191
7.2 Predicate Logic 193
7.2.1 Interpretations of Predicate Logic Formula 195
7.2.2 Inference in Predicate Logic 196
7.3 Fuzzy Logic 198
7.3.1 Fuzzy Quantifiers 201
7.3.2 Fuzzy Inference 202
lA Fuzzy Rule based System 204
7.5 Defuzziflcation Methods 205
7.6 Applications 210
7.6.1 Greg Viot's Fuzzy Cruise Controller 210
7.6.2 Air Conditioner Controller 216
Summary 219
Programming Assignment 220
Suggested Further Reading 220
References 220
Part 3 GENETIC ALGORITHMS
8 fundamentals OF GENETIC ALGORITHMS
8.1 Genetic Algorithms: History 227
8.2 Basic Concepts ^228
8.2.1 Biological Background 228
8.3 Creation of Offsprings 229
8.3.1 Search Space
8.4 Working Principle 230
8.5 Encoding 230
8.5.1 Binary Encoding 231
8.5.2 Octal Encoding (0 to 7) 234
8.5.3 Hexadecimal Encoding (0123456789ABCDEF) 234
8.5.4 Permutation Encoding 235
8.5.5 Value Encoding 236
8.5.6 Tree Encoding 236
8.6 Fitness Function 237
8.7 Reproduction 242
8.7.1 Roulette-wheel Selection 242
8.7.2 Boltzmann Selection 245
8.7.3 Tournament Selection 245
8.7.4 Rank Selection 247
8.7.5 Steady-state Selection 248
8.7.6 Elitism 248
8.7.7 Generation Gap and Steady-state Replacement 249
Summary 250
Programming Assignment 251
References 252
9 GENETIC MODELLING
9.1 Inheritance Operators 253
9.2 Cross Over 254
9.2.1 Single-site Cross Over 254
9.2.2 Two-point Cross Over 255
9.2.3 Multi-point Cross Over 255
9.2.4 Uniform Cross Over 256
9.2.5 Matrix Cross Over (Two-dimensional Cross Over) 257
9.2.6 Cross Over Rate 258
9.3 Inversion and Deletion 259
9.3.1 Inversion 259
9.3.2 Deletion and Duplication 259
9.3.3 Deletion and Regeneration 260
9.3.4 Segregation 260
9.3.5 Cross Over and Inversion 261
9.4 Mutation Operator 261
9.4.1 Mutation 261
9.4.2 Mutation Rate P„ 262
9.5 Bit-wise Operators 265
9.5.1 One's Complement Operator 263
9.5.2 Logical Bit-wise Operators 263
9.5.3 Shift Operators 264
9.6 Bit-wise Operators Used in GA 265
9.7 Generational Cycle 265
9.8 Convergence of Genetic Algorithm 271
9.9 Applications 272
9.9.1 Composite Laminates 272
9.9.2 Constrained Optimization 277
9.10 Multi-level Optimization 283
9.11 Real Life Problem 284
9.12 Differences and Similarities between GA and Other Traditional
Methods 287
9.13 Advances in GA 288
Summary 291
Programming Assignment 292
Suggested Further Reading 292
Some Useful Websites 293
References 293
Part 4 HYBRID SYSTEMS
10 INTEGRATION OF NEURAL NETWORKS, FUZZY LOGIC, AND GENETIC
ALGORITHMS
10.1 Hybrid Systems 298
10.1.1 Sequential Hybrid Systems 298
10.1.2 Auxiliary Hybrid Systems 298
10.1.3 Embedded Hybrid Systems 299
10.2 Neural Networks, Fuzzy Logic, and Genetic Algorithms Hybrids 300
10.2.1 Neuro-Fuzzy Hybrids 300
10.2.2 Neuro-Genetic Hybrids 300
10.2.3 Fuzzy-Genetic Hybrids 301
10.3 Preview of the Hybrid Systems to be Discussed 301
10.3.1 Genetic Algorithm based Backpropagation Network 302
10.3.2 Fuzzy-Backpropagation Network 302
10.3.3 Simplified Fuzzy ARTMAP 302
10.3.4 Fuzzy Associative Memories 302
10.3.5 Fuzzy Logic Controlled Genetic Algorithms 303
Summary 303
References 303
11 genetic ALGORITHM BASED BACKPROPAGATION NETWORKS
41.1 GA Based Weight Determination 306
11.1.1 Coding 306
11.1.2 Weight Extraction 308
11.1.3 Fitness Function 309
11.1.4 Reproduction 311
If. 1.5 Convergence 313
11.2 Applications 322
11.2.1 K-factor Determination in Columns 322
11.2.2 Electrical Load Forecasting 323
Summary 325
Programming Assignment 326
Suggested Further Reading 326
References 327
12 FUZZY BACKPROPAGATION NETWORKS
12.1 LR-type Fuzzy numbers 328
12.1.1 Operations on LR-type Fuzzy Numbers 330
12.2 Fuzzy Neuron 330
12.3 Fuzzy BP Architecture 331
12.4 Learning in Fuzzy BP 333
12.5 Inference by Fuzzy BP 339
12.6 Applications 347
12.6.1 Knowledge Base Evaluation 348
12.6.2 Earthquake damage Evaluation 353
Summary 355
Programming Assignment 356
References 357
13 SIMPLIFIED FUZZY ARTMAP
13.1 Fuzzy ARTMAP: A Brief Introduction 358
13.2 Simplified Fuzzy ARTMAP 359
13.2.1 Input Normalization 360
13.2.2 Output Node Activation 361
13.3 Working of Simplified Fuzzy ARTMAP 364
13.4 Application: Image Recognition 370
13.4.1 Feature Extraction—Moment Based Invariants 372
13.4.2 Computation of Invariants 375
13.4.3 Structure of the Simplified Fuzzy ARTMAP Based
Pattern Recognizer 380
13.4.4 Experimental Study 381
13.5 Recent Trends 384
Summary 384
Programming Assignment 385
References 387
14 FUZZY ASSOCIATIVE MEMORIES
14.1 FAM—An Introduction 389
14.2 Single Association FAM 390
14.2.1 Graphical Method of Inference 392
14.2.2 Correlation Matrix Encoding 393
14.3 Fuzzy Hebb FAMs 395
14.4 FAM Involving a Rule Base 400
14.5 FAM Rules with Multiple Antecedents/Consequents 401
14.5.1 Decomposition Rules 404
14.6 Applications 406
14.6.1 Balancing an Inverted Pendulum 406
14.6.2 Fuzzy Truck Backer-upper System 411
Summary 414
Programming Assignment 415
Suggested Further Reading 416
References 416
15 FUZZY LOGIC CONTROLLED GENETIC ALGORITHMS
15.1 Soft Computing Tools 417
15.1.1 Fuzzy Logic as a Soft Computing Tool 417
15.1.2 Genetic Algorithm as a Soft Computing Tool 418
15.2 Problem Description of Optimum Design 418
15.3 Fuzzy Constraints 419
15.4 Illustrations 420
15.4.1 Optimization of the Weight of a Beam 420
15.4.2 Optimal Mix Design for High Performance Concrete 422
15.5 GA in Fuzzy Logic Controller Design 424
15.6 Fuzzy Logic Controller 425
15.6.1 Components of a Fuzzy Logic Controller (FLC) 425
15.6.2 Fuzzy IF-THEN Rules 426
15.7 FLC-GA Based Structural Optimization 429
15.8 Applications 429
15.8.1 Optimum Truss 429
15.8.2 112 Bar Dome Space Truss 431
Summary 432
Programming Assignment 433
Suggested Further Reading 435
References 435