Welcome, guest | Sign In | My Account | Store | Cart

Apply the IAC (interactive-competition-and-activation) model to analyzing a database.

Python, 154 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
# Constants defining the neuron's response curve

minact, rest, thresh, decay, maxact = -0.2, -0.1, 0.0, 0.1, 1.0
alpha, gamma, estr = 0.1, 0.1, 0.4

units = []
pools = []
unitbyname = {}

class Unit(object):
    __slots__ = ['name', 'pool', 'extinp', 'activation', 'output', 'exciters', 'newact']
    def __init__(self, name, pool):
        self.name = name
        self.pool = pool
        self.reset()
        self.exciters = []
        unitbyname[name] = self
    def reset(self):
        self.setext(0.0)
        self._setactivation()
    def setext(self, weight=1.0):
        self.extinp = weight
    def _setactivation(self, val=rest):
        self.activation = val
        self.output = max(thresh, val)
    def addexciter(self, aunit):
        self.exciters.append(aunit)
    def remove(self, aunit):
        self.exciters.remove(aunit)
    def computenewact(self):
        ai = self.activation
        plus = sum(exciter.output for exciter in self.exciters)
        minus = self.pool.sum - self.output
        netinput = alpha*plus - gamma*minus + estr*self.extinp
        if netinput > 0:
            ai = (maxact-ai)*netinput - decay*(ai-rest) + ai
        else:
            ai = (ai-minact)*netinput - decay*(ai-rest) + ai
        self.newact = max(min(ai, maxact), minact)
    def commitnewact(self):
        self._setactivation(self.newact)

class Pool(object):
    __slots__ = ['sum', 'members']
    def __init__(self):
        self.sum = 0.0
        self.members = set()
    def addmember(self, member):
        self.members.add(member)
    def updatesum(self):
        self.sum = sum(member.output for member in self.members)
    def display(self):
        result = sorted(((unit.activation, unit.name) for unit in self.members), reverse=True)
        for i, (act, unitbyname) in enumerate(result):
            print '%s: %.2f\t' % (unitbyname, act),
            if i % 4 == 3: print
        print '\n'

def load(filename):
    """Load in a database and interpret it as a network

    First column must be unique keys which define the instance units.
    Each column is a pool (names, gangs, ages, etc).
    Every row is mutually excitory.
    """
    units[:] = []
    pools[:] = []
    for line in open(filename):
        relatedunits = line.split()
        if not len(relatedunits): continue
        key = len(units)
        for poolnum, name in enumerate(relatedunits):
            if poolnum >= len(pools):
                pools.append(Pool())
            pool = pools[poolnum]
            if name in unitbyname:
                unit = unitbyname[name]
            else:
                unit = Unit(name, pool)
                units.append(unit)
            pool.addmember(unit)
            if poolnum > 0:
                units[key].addexciter(unit)
                unit.addexciter(units[key])

def reset():
    for unit in units:
        unit.reset()

def depair(i, j):
    unitbyname[i].remove(unitbyname[j])
    unitbyname[j].remove(unitbyname[i])

def touch(itemstr, weight=1.0):
    for name in itemstr.split():
        unitbyname[name].setext(weight)

def run(times=100):
    """Run n-cycles and display result"""
    for i in xrange(times):
        for pool in pools:
            pool.updatesum()
        for unit in units:
            unit.computenewact()
        for unit in units:
            unit.commitnewact()
    print '-' * 20
    for pool in pools:
        pool.display()

if __name__ == '__main__' or 1:
    load('jets.txt')    
    touch('Ken', weight=0.8)
    run()

    reset()
    touch('Sharks 20 jh sing burglar')
    run()

    reset()
    touch('Lance')
    depair('Lance','burglar')
    run()

SampleFile = """
Art         Jets        40      jh      sing    pusher
Al          Jets        30      jh      mar     burglar
Sam         Jets        20      col     sing    bookie
Clyde       Jets        40      jh      sing    bookie
Mike        Jets        30      jh      sing    bookie
Jim         Jets        20      jh      div     burglar
Greg        Jets        20      hs      mar     pusher
John        Jets        20      jh      mar     burglar
Doug        Jets        30      hs      sing    bookie
Lance       Jets        20      jh      mar     burglar
George      Jets        20      jh      div     burglar
Pete        Jets        20      hs      sing    bookie
Fred        Jets        20      hs      sing    pusher
Gene        Jets        20      col     sing    pusher
Ralph       Jets        30      jh      sing    pusher

Phil        Sharks      30      col     mar     pusher
Ike         Sharks      30      jh      sing    bookie
Nick        Sharks      30      hs      sing    pusher
Don         Sharks      30      col     mar     burglar
Ned         Sharks      30      col     mar     bookie
Karl        Sharks      40      hs      mar     bookie
Ken         Sharks      20      hs      sing    burglar
Earl        Sharks      40      hs      mar     burglar
Rick        Sharks      30      hs      div     burglar
Ol          Sharks      30      col     mar     pusher
Neal        Sharks      30      hs      sing    bookie
Dave        Sharks      30      hs      div     pusher
"""

The example is taken from Parallel Distributed Processing, a classic text on neural networks.

The field values represent neurons.

The database rows represent excitory connections.

The database columns represent inhibitory connections.

The network is probed by applying a stimulus to a set of neurons.

The output is a sorted list of neurons and their activation levels.

The network can generalize from small datasets and is resiliant when information is missing.

3 comments

Nick Vatamaniuc 17 years, 7 months ago  # | flag

Why use __slots__? In general using __slots__ is not advisable unless there is a significant performance gain. __slots__ will cripple your classes and will cause strange side-effects later on, especially for somebody else who would use your code.

Raymond Hettinger (author) 17 years, 7 months ago  # | flag

Reasons for using __slots__. When applying this recipe to large databases, the use of __slots__ pays-off in two ways. First, it cuts memory use -- there is one Unit instance for every unique field value in the database -- each of those Units would need its own dictionary without __slots__. Second, the expense of propagation of activation and inhibition is markedly reduced when using psyco on code with __slots__ defined. If you don't like the __slots__ line, it is trivial for you to take it out.

Aschwin Wesselius 14 years, 11 months ago  # | flag

Hi, Maybe this is a late comment, but I just saw your presentation about AI just recently.

I'm more a PHP developer and was trying to port this code to PHP. And now I'm a bit stuck. Is there anyone who can help me with porting this?

Or maybe there is already a similar snippet somewhere that implements the same approach to a neural net as this one.

Thanks.

  • Unomi -
Created by Raymond Hettinger on Wed, 26 Jul 2006 (PSF)
Python recipes (4591)
Raymond Hettinger's recipes (97)

Required Modules

  • (none specified)

Other Information and Tasks