Welcome, guest | Sign In | My Account | Store | Cart

This is an attempt at emulating the algorithm from these scientific articles:

  1. 2011 - Artificial Astrocytes Improve Neural Network Performance
  2. 2012 - Computational Models of Neuron-Astrocyte Interactions Lead to Improved Efficacy in the Performance of Neural Networks

The objective of the program is to train a neural network to classify the four inputs (the dimensions of a flower) into one of three categories (three species of flower), (taken from the Iris Data Set from the UCI Machine Learning Repository). This program has two learning phases: the first is a genetic algorithm (supervised), the second is a neuroglial algorithm (unsupervised). This ANGN is a development of a previous program only consisting of a genetic algorithm which can be found here.

The second phase aims to emulate astrocytic interaction with neurons in the brain. The algorithm is based on two axioms: a) astrocytes are activated by persistent neuronal activity b) astrocytic effects occur over a longer time-scale than neurons. Each neuron has an associated astrocyte which counts the number of times its associated neuron fires (+1 for active -1 for inactive). If the counter reaches its threshold (defined as Athresh) the astrocyte is activated and for the next x iterations (defined as Adur) the astrocyte modifies the incoming weights to that particular neuron. If the counter reached a maximum due to persistent firing the incoming weights are increase by 25% for the proceeding Adur iterations; conversely if the counter reached a minimum due to persistent lack of firing the weights are decreased by 50% for the following Adur iterations). For a detailed description of the algorithm see the linked articles. For a general understanding of how this program was coded look at the pseudo-code/schematic here.

Any comments for improvements are welcome. There are several issues in this program which require addressing, please scroll down below code to read about these issues.

Python, 642 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
from operator import itemgetter, attrgetter
import math
from math import copysign
from random import *
import timeit
from timeit import Timer as t
from matplotlib.pyplot import *
from numpy import *

def sigmoid (x):
  return math.tanh(x)

class NN:
  # ni,nh,no = n of input (i), hidden (h) and output (o) nodes
  # ai,ah,ao = output value for nodes in i,h,o layers
  # Ah, Ao = Astrocyte activation value/counter of partner neuronal activity (h,o layers)
  # Ah_countdown = counts down duration of astrocyte action in itererations
  # wi,wo = input and output weights initialized to random values
  def __init__(self, NI, NH, NO):
    self.ni = NI
    self.nh = NH
    self.no = NO
    self.ai = ones(ni)
    self.ah = ones(nh) 
    self.ao = ones(no) 
    self.Ah = [0]*self.nh 
    self.Ao = [0]*self.no 
    self.Ah_countdwn = [0]*self.nh
    self.Ao_countdwn = [0]*self.no
    self.wi = random.uniform(-2,2,(ni,nh))
    self.wo = random.uniform(-2,2,(nh,no))

  def runNN (self, inputs):
    if len(inputs) != self.ni:
      print 'incorrect number of inputs'
    for i in range(self.ni):
      self.ai[i] = inputs[i]
    for j in range(self.nh):
      self.ah[j] = sigmoid(sum([ self.ai[i]*self.wi[i][j] for i in range(self.ni) ]))     
    for k in range(self.no):
      self.ao[k] = sigmoid(sum([ self.ah[j]*self.wo[j][k] for j in range(self.nh) ]))
    return self.ao
    
  def runNGA (self, inputs):
    if len(inputs) != self.ni:
      print 'incorrect number of inputs'
    for i in range(self.ni):
      self.ai[i] = inputs[i]
      
    for j in range(self.nh):
      self.ah[j] = sigmoid(sum([ self.ai[i] * self.wi[i][j] for i in range(self.ni) ]))
      # check neuron activation
      if self.ah[j] > 0: self.Ah[j] +=1
      else: self.Ah[j] -=1

      # check if astrocyte should be activated
      if self.Ah[j] >= Athresh:
        self.Ah_countdwn[j] = Adur
        self.Ah[j] = 0
      elif self.Ah[j] <= -Athresh:
        self.Ah_countdwn[j] = -Adur
        self.Ah[j] = 0
        
      # check if astrocyte is active --> perform actions
      if self.Ah_countdwn[j] > 0:
        for i in range(ni):
          if not self.wi[i][j] > wl:
            self.wi[i][j] += (wl-self.wi[i][j])*0.25
        self.Ah_countdwn[j] -= 1
      elif self.Ah_countdwn[j] < 0:
        for i in range(ni):
          if not self.wi[i][j] < -wl:
            self.wi[i][j] -= (wl+self.wi[i][j])*0.5
        self.Ah_countdwn[j] += 1
        
    for k in range(self.no):
      self.ao[k] = sigmoid(sum([ self.ah[j] * self.wo[j][k] for j in range(self.nh) ]))
      # check neuron activation
      if self.ao[k] > 0: self.Ao[k] += 1
      else: self.Ao[k] -= 1

      # check if astrocyte should be activated
      if self.Ao[k] >= Athresh: 
        self.Ao_countdwn[k] = Adur
        self.Ao[k] = 0
      elif self.Ao[k] <= -Athresh: 
        self.Ao_countdwn[k] = -Adur
        self.Ao[k] = 0
      
      # check if astrocyte is active --> perform actions
      if self.Ao_countdwn[k] > 0:
        for j in range(nh):
          if not self.wo[j][k] > wl:
            self.wo[j][k] += (wl-self.wo[j][k])*0.25
        self.Ao_countdwn[k] -= 1
      elif self.Ao_countdwn[k] < 0:
        for j in range(nh):
          if not self.wo[j][k] < -wl:
            self.wo[j][k] -= (wl+self.wo[j][k])*0.5
        self.Ao_countdwn[k] += 1

  def test(self, patterns):
    results, targets = [], []
    for p in patterns:
      inputs = p[0]
      rounded = [ round(i) for i in self.runNN(inputs) ]
      if rounded == p[1]: result = '+++++'
      else: result = '-----'
      print '%s %s %s %s %s %s %s' %( 'Inputs:', p[0], '-->', str([ round(i,3) for i in self.runNN(inputs) ]).rjust(65), 'Target', p[1], result)
      results+= eval(repr([self.runNN(inputs)]))
      targets += [p[1]]
    return results, targets

  def sumErrors (self):
    error = 0.0
    for p in pat:
      inputs = p[0]
      targets = p[1]
      self.runNN(inputs)
      error += self.calcError(targets)
    inverr = 1.0/error
    return inverr

  def calcError (self, targets):
    error = 0.0
    for k in range(len(targets)):
      error += 0.5 * (targets[k]-self.ao[k])**2
    return error

  def assignWeights (self, weights, I):
    io = 0
    for i in range(self.ni):
      for j in range(self.nh):
        self.wi[i][j] = weights[I][io][i][j]
    io = 1
    for j in range(self.nh):
      for k in range(self.no):
        self.wo[j][k] = weights[I][io][j][k]

  def testWeights (self, weights, I):
    same = []
    io = 0
    for i in range(self.ni):
      for j in range(self.nh):
        if self.wi[i][j] != weights[I][io][i][j]:
          same.append(('I',i,j, round(self.wi[i][j],2),round(weights[I][io][i][j],2),round(self.wi[i][j] - weights[I][io][i][j],2)))
    io = 1
    for j in range(self.nh):
      for k in range(self.no):
        if self.wo[j][k] !=  weights[I][io][j][k]:
          same.append((('O',j,k), round(self.wo[j][k],2),round(weights[I][io][j][k],2),round(self.wo[j][k] - weights[I][io][j][k],2)))
    if same != []:
      print I,'\n',same

def roulette (fitnessScores):
  cumalativeFitness = 0.0
  r = random.random()
  for i in range(len(fitnessScores)): 
    cumalativeFitness += fitnessScores[i]
    if cumalativeFitness > r: 
      return i
      
def calcFit (numbers):  # each fitness is a fraction of the total error
  total, fitnesses = sum(numbers), []
  for i in range(len(numbers)):           
    fitnesses.append(numbers[i]/total)
  return fitnesses

# Takes a population of NN objects, returns a list of the weights tupled with their 
# fitness scores.
# Calculates the MSE for each instance by subtraction from target value and sums over all
# instances. 
# Fitness score for each individual is a fraction of the total error of the population.
def pairPop (pop):
  weights, errors = [], []
  for i in range(len(pop)):                 
    weights.append([pop[i].wi,pop[i].wo])   
    errors.append(pop[i].sumErrors())       
  fitnesses = calcFit(errors)               
  for i in range(int(pop_size*0.15)): 
    print str(i).zfill(2), '1/sum(MSEs)', str(errors[i]).rjust(15), str(int(errors[i]*graphical_error_scale)*'-').rjust(20), 'fitness'.rjust(12), str(fitnesses[i]).rjust(17), str(int(fitnesses[i]*1000)*'-').rjust(20)
  print 
  del pop
  return zip(weights, errors, fitnesses)
  
# Takes weights for a pop and ranks the weights 
# Returns the weights paired and sorted in order of fitness
def rankPop (newpopW):
  print 'RANKPOP START'
  pop, errors, copy = [ NN(ni,nh,no) for i in range(pop_size) ], [], []
  for i in range(pop_size): copy.append(newpopW[i])
  for i in range(pop_size):  
    pop[i].assignWeights(newpopW, i)                                    # each individual is assigned the weights generated from previous iteration
    pop[i].testWeights(newpopW, i)
  for i in range(pop_size):  
    pop[i].testWeights(newpopW, i)
  pairedPop = pairPop(pop)                                              # the fitness of these weights is calculated and tupled with the weights
  rankedPop = sorted(pairedPop, key = itemgetter(-1), reverse = True)   # weights are sorted in descending order of fitness (fittest first)
  errors = [ eval(repr(x[1])) for x in rankedPop ]
  print 'END RANKPOP'
  return rankedPop, eval(repr(rankedPop[0][1])), float(sum(errors))/float(len(errors))

def iteratePop (rankedPop):
  rankedWeights = [ item[0] for item in rankedPop]
  fitnessScores = [ item[-1] for item in rankedPop]
  newpopW = [ eval(repr(x)) for x in rankedWeights[:int(pop_size*0.15)] ]
  while len(newpopW) <= pop_size:                                       # Breed two randomly selected but different chromos until pop_size reached
    ch1, ch2 = [], []
    index1 = roulette(fitnessScores)                                    
    index2 = roulette(fitnessScores)
    while index1 == index2:                                             # ensures different chromos are used for breeeding 
      index2 = roulette(fitnessScores)
    ch1.extend(eval(repr(rankedWeights[index1])))
    ch2.extend(eval(repr(rankedWeights[index2])))
    if random.random() < crossover_rate: 
      ch1, ch2 = crossover(ch1, ch2)
    mutate(ch1)  
    mutate(ch2)  
    newpopW.append(ch1)
    newpopW.append(ch2)
  return newpopW

def NGA(newpopW):
  print 'NGA START'
  pop = [ NN(ni,nh,no) for i in range(pop_size) ]
  #pop = [ NN(ni,nh,no) ]*pop_size
  weights = []
  for i in range(pop_size):
    pop[i].assignWeights(newpopW, i)
    pop[i].testWeights(newpopW, i)
    for p in range(len(pat)):
      inputs, targets = pat[p][0], pat[p][1]
      for m in range(m_iters):
        pop[i].runNGA(inputs)
    weights.append([pop[i].wi,pop[i].wo])
  del pop
  print 'END NGA'
  return weights

graphical_error_scale = 500
max_iterations = 100
pop_size = 100
mutation_rate = 0.1
crossover_rate = 0.8
ni, nh, no = 4,6,3 # number of input (i), hidden (h) and output (o) nodes
m_iters = 6 # number of iterations per instance
Athresh = 3 # threshold count for activation of astrocyte
Adur = 2 # duration of action of astrocyte
wl = 2.0 # weight limit

def main ():
  
  # Generates a population of NNs with random weights and rank them based on their error
  pop = [ NN(ni,nh,no) for i in range(pop_size) ] 
  pairedPop = pairPop(pop)
  rankedPop = sorted(pairedPop, key = itemgetter(-1), reverse = True)
  
  # Keep iterating new pops until max_iterations
  iters = 0
  tops, avgs = [], []
  newpopW = iteratePop(rankedPop)
  
  while iters != max_iterations:
  
    if iters%1 == 0:
      print 'Iteration'.rjust(150), iters

    newpopW = NGA(newpopW)
    rankedPop, toperr, avgerr = rankPop(newpopW)
    newpopW = iteratePop(rankedPop)

    tops.append(toperr)
    avgs.append(avgerr)
    iters+=1
  
  # test a NN with the fittest weights
  tester = NN (ni,nh,no)
  fittestWeights = [ x[0] for x in rankedPop ]
  tester.assignWeights(fittestWeights, 0)
  results, targets = tester.test(testpat)
  plotresults(results,targets)
  
def plotresults(results,targets):
  for i in range(len(results)): 
    print results[i], targets[i]
  lp = len(pat)
  x = arange(0,lp/3)
  title2 = 'Test after '+str(iters)+' iterations'
  title(title2)
  for i in range(3):
    sp = '3'+'1'+str(i)
    subplot(int(sp))
    ylabel('Node output')
    xlabel('Instances')
    for x in range(lp/3):
      plot( results[x+(i*(lp/3))], label='Instance'+str(x))
      print results[x+(i*(lp/3))]
    plot( targets[i*(lp/3)], 'kx-', markersize = 10, linewidth = 3)
  legend(loc = 'upper right')
  annotate(s='Target Values', xy = (110, 0),color = 'black', family = 'sans-serif', size  ='small')
  annotate(s='Test Values', xy = (110, 0.5),color = 'red', family = 'sans-serif', size  ='small', weight = 'bold')
  figure(2)
  title('Top individual error evolution')
  title('Population average error evolution')
  plot( avgs, '-g', linewidth = 0.5)
  plot( tops, '-r', linewidth = 2)
  ylabel('Inverse error')
  xlabel('Iterations')
  show()
  
  print 'max_iterations',max_iterations,'\tpop_size',pop_size,'pop_size*0.15',int(pop_size*0.15),'\tmutation_rate',mutation_rate,'crossover_rate',crossover_rate,'ni, nh, no',ni, nh, no
  

def crossover (m1, m2):
  r = random.randint(0, (ni*nh)+(nh*no) ) # ni*nh+nh*no = total n of weights
  output1 = [ [[0.0]*nh]*ni ,[[0.0]*no]*nh ]
  output2 = [ [[0.0]*nh]*ni ,[[0.0]*no]*nh ]
  for i in range(len(m1)):
    for j in range(len(m1[i])):
      for k in range(len(m1[i][j])):
        if r >= 0:
          output1[i][j][k] = m1[i][j][k]
          output2[i][j][k] = m2[i][j][k]
        elif r < 0:
          output1[i][j][k] = m2[i][j][k]
          output2[i][j][k] = m1[i][j][k]
        r -=1
  return output1, output2

def mutate (m):
  for i in range(len(m)):
    for j in range(len(m[i])):
      for k in range(len(m[i][j])):
        if random.random() < mutation_rate:
            m[i][j][k] = random.uniform(-2.0,2.0)

pat = [
  [[5.1, 3.5, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.0, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.7, 3.2, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.1, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.6, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.9, 1.7, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.4, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.4, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 2.9, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.7, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.4, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.0, 1.4, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[4.3, 3.0, 1.1, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.8, 4.0, 1.2, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.7, 4.4, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.9, 1.3, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.5, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.7, 3.8, 1.7, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.5, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.4, 1.7, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.7, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.6, 1.0, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.3, 1.7, 0.5], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.4, 1.9, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.0, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.4, 1.6, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 3.5, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 3.4, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.7, 3.2, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.1, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.4, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 4.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.5, 4.2, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.2, 1.2, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.5, 3.5, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 3.0, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.4, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.5, 1.3, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[4.5, 2.3, 1.3, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 3.2, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.5, 1.6, 0.6], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.9, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.0, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.2, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.3, 3.7, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.3, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[7.0, 3.2, 4.7, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.4, 3.2, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.9, 3.1, 4.9, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.3, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.5, 2.8, 4.6, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.8, 4.5, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 3.3, 4.7, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[4.9, 2.4, 3.3, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.6, 2.9, 4.6, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.2, 2.7, 3.9, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.0, 2.0, 3.5, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.9, 3.0, 4.2, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.2, 4.0, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.9, 4.7, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.9, 3.6, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.1, 4.4, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 3.0, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.7, 4.1, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.2, 2.2, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.5, 3.9, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.9, 3.2, 4.8, 1.8], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.8, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 2.5, 4.9, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.8, 4.7, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.4, 2.9, 4.3, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.6, 3.0, 4.4, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.8, 2.8, 4.8, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.0, 5.0, 1.7], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.9, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.6, 3.5, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.4, 3.8, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.4, 3.7, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.7, 3.9, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.7, 5.1, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.4, 3.0, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 3.4, 4.5, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.1, 4.7, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 2.3, 4.4, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 3.0, 4.1, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.5, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.6, 4.4, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 3.0, 4.6, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.6, 4.0, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.0, 2.3, 3.3, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.7, 4.2, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 3.0, 4.2, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.9, 4.2, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.2, 2.9, 4.3, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.1, 2.5, 3.0, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.8, 4.1, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 3.3, 6.0, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.7, 5.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[7.1, 3.0, 5.9, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.9, 5.6, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.8, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[7.6, 3.0, 6.6, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[4.9, 2.5, 4.5, 1.7], [0, 0, 1], ['Iris-virginica']] ,
  [[7.3, 2.9, 6.3, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 2.5, 5.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.6, 6.1, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.2, 5.1, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.7, 5.3, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.8, 3.0, 5.5, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[5.7, 2.5, 5.0, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.8, 5.1, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 3.2, 5.3, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.5, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 3.8, 6.7, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 2.6, 6.9, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.0, 2.2, 5.0, 1.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.2, 5.7, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.6, 2.8, 4.9, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 2.8, 6.7, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.7, 4.9, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.3, 5.7, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.2, 6.0, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.2, 2.8, 4.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.1, 3.0, 4.9, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.8, 5.6, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.0, 5.8, 1.6], [0, 0, 1], ['Iris-virginica']] ,
  [[7.4, 2.8, 6.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[7.9, 3.8, 6.4, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.8, 5.6, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.8, 5.1, 1.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.1, 2.6, 5.6, 1.4], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 3.0, 6.1, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 3.4, 5.6, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 3.1, 5.5, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.0, 3.0, 4.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.1, 5.4, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.1, 5.6, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.1, 5.1, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.7, 5.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.8, 3.2, 5.9, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.3, 5.7, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.0, 5.2, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.5, 5.0, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.2, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.2, 3.4, 5.4, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.9, 3.0, 5.1, 1.8], [0, 0, 1], ['Iris-virginica']]
]
testpat = [
  [[5.1, 3.5, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.0, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.7, 3.2, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.1, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.6, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.9, 1.7, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.4, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.4, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 2.9, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.7, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.4, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.0, 1.4, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[4.3, 3.0, 1.1, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.8, 4.0, 1.2, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.7, 4.4, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.9, 1.3, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.5, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.7, 3.8, 1.7, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.5, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.4, 1.7, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.7, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.6, 1.0, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.3, 1.7, 0.5], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.4, 1.9, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.0, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.4, 1.6, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 3.5, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 3.4, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.7, 3.2, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.1, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.4, 3.4, 1.5, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[5.2, 4.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.5, 4.2, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.2, 1.2, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.5, 3.5, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.9, 3.1, 1.5, 0.1], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 3.0, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.4, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.5, 1.3, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[4.5, 2.3, 1.3, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[4.4, 3.2, 1.3, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.5, 1.6, 0.6], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.9, 0.4], [1, 0, 0], ['Iris-setosa']] ,
  [[4.8, 3.0, 1.4, 0.3], [1, 0, 0], ['Iris-setosa']] ,
  [[5.1, 3.8, 1.6, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[4.6, 3.2, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.3, 3.7, 1.5, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[5.0, 3.3, 1.4, 0.2], [1, 0, 0], ['Iris-setosa']] ,
  [[7.0, 3.2, 4.7, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.4, 3.2, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.9, 3.1, 4.9, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.3, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.5, 2.8, 4.6, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.8, 4.5, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 3.3, 4.7, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[4.9, 2.4, 3.3, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.6, 2.9, 4.6, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.2, 2.7, 3.9, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.0, 2.0, 3.5, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.9, 3.0, 4.2, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.2, 4.0, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.9, 4.7, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.9, 3.6, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.1, 4.4, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 3.0, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.7, 4.1, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.2, 2.2, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.5, 3.9, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.9, 3.2, 4.8, 1.8], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.8, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 2.5, 4.9, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 2.8, 4.7, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.4, 2.9, 4.3, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.6, 3.0, 4.4, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.8, 2.8, 4.8, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.0, 5.0, 1.7], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.9, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.6, 3.5, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.4, 3.8, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.4, 3.7, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.7, 3.9, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 2.7, 5.1, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.4, 3.0, 4.5, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.0, 3.4, 4.5, 1.6], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.7, 3.1, 4.7, 1.5], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 2.3, 4.4, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 3.0, 4.1, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.5, 4.0, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.5, 2.6, 4.4, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.1, 3.0, 4.6, 1.4], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.8, 2.6, 4.0, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.0, 2.3, 3.3, 1.0], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.6, 2.7, 4.2, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 3.0, 4.2, 1.2], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.9, 4.2, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.2, 2.9, 4.3, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.1, 2.5, 3.0, 1.1], [0, 1, 0], ['Iris-versicolor']] ,
  [[5.7, 2.8, 4.1, 1.3], [0, 1, 0], ['Iris-versicolor']] ,
  [[6.3, 3.3, 6.0, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.7, 5.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[7.1, 3.0, 5.9, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.9, 5.6, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.8, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[7.6, 3.0, 6.6, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[4.9, 2.5, 4.5, 1.7], [0, 0, 1], ['Iris-virginica']] ,
  [[7.3, 2.9, 6.3, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 2.5, 5.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.6, 6.1, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.2, 5.1, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.7, 5.3, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.8, 3.0, 5.5, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[5.7, 2.5, 5.0, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.8, 5.1, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 3.2, 5.3, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.5, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 3.8, 6.7, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 2.6, 6.9, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.0, 2.2, 5.0, 1.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.2, 5.7, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.6, 2.8, 4.9, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 2.8, 6.7, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.7, 4.9, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.3, 5.7, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.2, 6.0, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.2, 2.8, 4.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.1, 3.0, 4.9, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.8, 5.6, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[7.2, 3.0, 5.8, 1.6], [0, 0, 1], ['Iris-virginica']] ,
  [[7.4, 2.8, 6.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[7.9, 3.8, 6.4, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 2.8, 5.6, 2.2], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.8, 5.1, 1.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.1, 2.6, 5.6, 1.4], [0, 0, 1], ['Iris-virginica']] ,
  [[7.7, 3.0, 6.1, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 3.4, 5.6, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.4, 3.1, 5.5, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.0, 3.0, 4.8, 1.8], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.1, 5.4, 2.1], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.1, 5.6, 2.4], [0, 0, 1], ['Iris-virginica']] ,
  [[6.9, 3.1, 5.1, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.8, 2.7, 5.1, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.8, 3.2, 5.9, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.3, 5.7, 2.5], [0, 0, 1], ['Iris-virginica']] ,
  [[6.7, 3.0, 5.2, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[6.3, 2.5, 5.0, 1.9], [0, 0, 1], ['Iris-virginica']] ,
  [[6.5, 3.0, 5.2, 2.0], [0, 0, 1], ['Iris-virginica']] ,
  [[6.2, 3.4, 5.4, 2.3], [0, 0, 1], ['Iris-virginica']] ,
  [[5.9, 3.0, 5.1, 1.8], [0, 0, 1], ['Iris-virginica']]
]
if __name__ == "__main__":
    main()

Fundamental Algorithm Logic Issues

Program does not learn Ultimately the program does not improve over each iteration and the test accuracy after all iterations is often worse than random weights, in addition it is much slower than a simple genetic algorithm which I used as a basis to build this program. Here are some potential problems with the algorithm.

  1. Unnecessary iterations The astrocyte requires Athresh number of iterations before it can be activated. However during the first Athresh number of iterations of m_iters no weights are changing as no astrocytes can be active yet and therefore there is no need for these iterations. These could be condensed into one iteration.
  2. Uniform inputs Since inputs are dimensions of a flower they are continuous values. Most values are above 2 and since weights are limited to +/-2 the tanh() value will mostly be above 0. Therefore neurons will normally fire and astrocytes will normally increase their activity and therefore normally increase incoming weights by 0.25% until weight limit is reached.
  3. Increasing and decreasing weights by percentage I have used the formula self.wi[i][j] += (wl-self.wi[i][j])*0.25 and self.wi[i][j] -= (wl+self.wi[i][j])*0.5 for increasing and decreasing the weights respectively. Where wl is the weight limit and self.wi[i][j] is the weight in question. I introduced the wl so that the weights can actually become negative if they were originally positive and become positive if they were originally negative. In this way the weights approach the maximum and minimum weight limits rather than approaching 0 , infinity or negative-infinity.
  4. Order of input patterns Since the astrocyte activity is unmodified from the last input pattern each time a new input pattern is presented to the individual network it may be important in what order the input patterns are presented to the network.
  5. Input layer lack astrocytes Unlike as is implied in the schematics of the scientific articles I have not given the input neurons astrocytes as these will always be activated since the inputs are always above 0.
  6. Modification of input weights only In the scientific article it is unclear whether the input weights or the output weights or both weights of the neuron should be modified

Python Issues

Matrices duplicate problems (These problems may be a result of me not understanding fully the nature of python objects). On several occassions I have had problems with values from matrices becoming linked copies of the original values rather than just straight copies of the value. Thus when modifying the copy it has modified the original also and/or produced strange results. I have worked around these problems as follows however no doubt this has made my program much slower:

  1. Fresh pop required When assigning the weights to an individual assignWeights(self, inputs) it is required that I use a freshly created NN object rather than recycling the NN objects from the old pop. Otherwise the weights will change after the assignment for an unkown reason and produce a different error.
  2. Iterating new pop During the function iteratePop(rankedPop) it is required that I use eval(repr()) when selecting an individual for the new population otherwise the selected individual will be a linked copy to the original. This is a problem because if the copy is modified by mutation or crossover it will modify the original and if I try to select the original individual again for another individual for the new population it will no longer have its original values matching its fitness score. eval(repr()) considerably slows down my program.
  3. numpy These first two changes above were sufficient for the functioning of the genetic algorithm alone, however upon the addition of the NGA I encountered furthermore problems to do with duplicates and incorrect assigning of weights in my results so I decided to change the weights from a list of floats to numpy arrays. This seems to resolved the remaining issues but is now a much much slower program!