Welcome, guest | Sign In | My Account | Store | Cart

Use reflection to generate lists of Python symbols. Use them to build a syntax coloring file for your preferred programmer's editor.

This is easier and less error-prone than updating your syntax file by reading "What's New In Version x" and other documentation.

Python, 159 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
# GenEditorSymbolLists.py
"""Make lists of Python symbols for a programmer's editor syntax coloring file

Redirect this program's stdout to a text file.  Cut and paste some or all of
the lists to build a syntax coloring file for your prefered programmer's
editor.

Copyright 2003 by James M Jinkins - Released to the Public Domain

Jim -dot_ Jinkins -at_ ComPorts -dot_ com
"""

import keyword
import types
import re

COMMENT_CHR = ";"   # Set to the comment indicator of the editor program

specialNamePat = re.compile("^.*(__([a-zA-Z0-9]+(_[a-zA-Z0-9]+)*)__).*$")
methodPat = re.compile("^<(slot wrapper|method|built-in method) .+>$")
instancePat = re.compile("^<.+ (instance at) .+>$")

def sortRemoveDupes(lst):
    """Sort the list, and remove duplicate symbols.
    """
    if len(lst) == 0:
        return lst
    lst.sort()
    lst = [lst[0]] + [lst[i] for i in range(1, len(lst))
            if lst[i] != lst[i - 1]]
    return lst

def printList(lst, title):
    """Write a title line.  List the symbols in the list on separate lines.
    """
    print "\n\n\n%s %s %s %s symbols %s" % (COMMENT_CHR * 6, title,
            COMMENT_CHR * 3, len(lst), COMMENT_CHR * 6)
    if len(lst) == 0:
        print "%s%s%s" % (COMMENT_CHR * 8, "empty list", COMMENT_CHR * 8)
    else:
        for le in lst:
            print le

##############################################################################

if __name__ == "__main__":
    allKeywords = keyword.kwlist
    allKeywords.sort()

    allTypes = dir(types)
    allTypes.sort()

    allBuiltins = dir(__builtins__)
    allBuiltins.sort()
    # Split allBuiltins
    builtinExceptions = []
    builtinNames = []
    for attrName in allBuiltins:
        if attrName in ["license", "copyright", "credits", "help"]:
            builtinNames.append(attrName)
        else:
            try:
                obj = eval("%s()" % attrName)
                if isinstance(obj, Exception):
                    builtinExceptions.append(attrName)
                else:
                    builtinNonExceptions.append(attrName)
            except Exception, e:
                builtinNames.append(attrName)

    normalTypeMethods = []
    specialTypeMethods = []
    normalTypeAttrs = []
    specialTypeAttrs = []
    for typeName in dir(types):
        try:
            typeDir = dir(eval("types.%s" % typeName))
            for attrName in typeDir:
                try:
                    attr = eval("types.%s.%s" % (typeName, attrName))
                    attrRepr = repr(attr)
                    # print "        attrRepr = %s"% attrRepr
                    m = methodPat.match(attrRepr)
                    if m:
                        m = specialNamePat.match(attrName)
                        if m:
                            specialTypeMethods.append(attrName)
                        else:
                            normalTypeMethods.append(attrName)
                    else:
                        m = specialNamePat.match(attrName)
                        if m:
                            specialTypeAttrs.append(attrName)
                        else:
                            normalTypeAttrs.append(attrName)
                except Exception, e:
                    pass
                    # print "    attr=%s - %s" % (attr, e)
        except Exception, e:
            pass
            # print "    No dir -- %s" % e

    # These special methods are not in the list, because neither the object
    # class nor any standard types class needs them.
    specialTypeMethods.extend(["__complex__", "__del__"])

    normalObjectMethods = []
    specialObjectMethods = []
    normalObjectAttrs = []
    specialObjectAttrs = []
    obj = object()
    for attrName in dir(object):
        # print "attrName = %s" % attrName
        try:
            attr = eval("object.%s" % attrName)
            attrRepr = repr(attr)
            # print "    attrRepr = %s" % attrRepr
            m = methodPat.match(attrRepr)
            if m:
                m = specialNamePat.match(attrName)
                if m:
                    specialObjectMethods.append(attrName)
                else:
                    normalObjectMethods.append(attrName)
            else:
                m = specialNamePat.match(attrName)
                if m:
                    specialObjectAttrs.append(attrName)
                else:
                    normalObjectAttrs.append(attrName)
        except Exception, e:
            print "    object.%s  Exception %s" % (attrName, e)

    specialMethods = sortRemoveDupes(specialTypeMethods + specialObjectMethods)
    normalMethods = sortRemoveDupes(normalTypeMethods + normalObjectMethods)
    specialAttrs = sortRemoveDupes(specialTypeAttrs + specialObjectAttrs)
    normalAttrs = sortRemoveDupes(normalTypeAttrs + normalObjectAttrs)

    # Remove duplicate symbols
    for x in specialAttrs[:]:
        if x in specialMethods:
            specialAttrs.remove(x)
    for x in builtinNames[:]:
        if x in specialAttrs:
            builtinNames.remove(x)
    for x in allTypes[:]:
        if x in specialAttrs:
            allTypes.remove(x)

    printList(allKeywords, "Keywords")
    # printList(allBuiltins, "All builtin names")
    printList(builtinExceptions, "Builtin exception classes")
    printList(builtinNames,
            "Other Builtin names - Functions and other objects")
    printList(specialAttrs, "Special attributes of types and object")
    printList(specialMethods, "Special methods of types and object")
    printList(normalMethods, "Other methods of types and object")
    printList(normalAttrs, "Other attributes of types and object")
    printList(allTypes, "Type names")

After I updated the syntax coloring file for my editor for Python 2.3, I worried that I had not found every new keyword, builtin function, or special method name.

After I wrote this program, I found that I had not. I rebuilt the file by cutting and pasting from the program's output.

To use GenEditorSymbolLists.py redirect its output to a text file. Cut and paste some or all of the lists of symbols into the syntax coloring file of your preferred text editor.

When I compared the file I created to the manually generated version, it had several more special method names and additional lists of methods and attributes from the object class and other standard classes in the types module. It was missing two special methods from the old file, __complex__ and __del__. They are in the Python documentation, but none of the standard classes uses them.

Additional code includes __complex__ and __del__ in the generated output. If you find any other symbols like these, please let me know.

1 comment

Jeremy Sproat 13 years, 11 months ago  # | flag

Due to changes in recent Python versions, I needed to change line 59 to this:

if attrName in ["license", "copyright", "credits", "help", "input", "raw_input", "exit", "quit"]:

...adding the input and exit functions so they don't get called during the script.