ActiveState Code

Recipe 576534: Dump Oracle db schema to text


Export Oracle schema to text. Usable to compare databases that should be the same

Oracle schema info: http://www.eveandersson.com/writing/data-model-reverse-engineering

Python
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
#!/usr/bin/python
# -*- coding: cp1250 -*-
__version__ = '$Id: schema_ora.py 654 2009-10-14 06:11:58Z mn $'

# export Oracle schema to text
# usable to compare databases that should be the same
#
# Oracle schema info:
# http://www.eveandersson.com/writing/data-model-reverse-engineering
#
#
# author: Michal Niklas

USAGE = 'usage:\n\tschema_ora.py username passwd tnsentry'

import sys

USE_JYTHON = 0

try:
	from com.ziclix.python.sql import zxJDBC
	USE_JYTHON = 1
	USAGE = """usage:
\tschema_ora.py jdbcurl user passwd
example:
\tjython schema_ora.py jdbc:oracle:thin:user/passwd@127.0.0.1:1521:dbname user passwd > db.schema 2> db.err
"""
except:
	USAGE = 'usage:\n\tschema_ora.py user passwd tnsentry'
	USE_JYTHON = 0
	import cx_Oracle

OUT_ENCODINGS = ('cp1250', 'iso8859_2')

TABLE_NAMES_SQL = """SELECT DISTINCT table_name
FROM user_tab_columns
WHERE INSTR(table_name, 'X_') <> 1
ORDER BY 1
"""


TABLE_INFO_SQL = """SELECT table_name, column_name, data_type, nullable,
decode(default_length, NULL, 0, 1) hasdef,
decode(data_type,
	'DATE', 11,
	'NUMBER', nvl(data_precision,38)||'.'||data_scale,
	data_length) data_length
FROM user_tab_columns
WHERE INSTR(table_name, 'X_') <> 1
ORDER BY 1, 2
"""

PRIMARY_KEYS_INFO_SQL = """SELECT uc.table_name, ucc.column_name
FROM user_constraints uc, user_cons_columns ucc
WHERE uc.constraint_name = ucc.constraint_name
AND uc.constraint_type = 'P'
AND INSTR(uc.table_name, 'X_') <> 1
ORDER BY uc.table_name, ucc.position"""


INDEXES_INFO_SQL = """SELECT ui.table_name, ui.index_name, ui.uniqueness
FROM user_indexes ui
WHERE INSTR(ui.table_name, 'X_') <> 1
ORDER BY 1, 2"""


INDEXES_COLUMNS_INFO_SQL = """SELECT table_name, column_name, index_name, column_position, descend
FROM user_ind_columns
WHERE INSTR(table_name, 'X_') <> 1
ORDER BY 1, 2, 3"""


COMPOSITE_INDEXES_COLUMNS_INFO_SQL = """SELECT table_name, column_name, index_name, column_position
FROM user_ind_columns
WHERE index_name in (select distinct index_name from USER_IND_COLUMNS where column_position > 1)
AND INSTR(table_name, 'X_') <> 1
ORDER BY table_name, index_name, column_position
"""


FOREIGN_KEYS_INFO_SQL = """SELECT uc.table_name
,ucc.column_name
,fc.table_name
,fc.constraint_name
FROM   user_cons_columns ucc
,user_constraints fc
,user_constraints uc
WHERE  uc.constraint_type = 'R'
AND    uc.constraint_name = ucc.constraint_name
AND    fc.constraint_name = uc.r_constraint_name
AND INSTR(uc.table_name, 'X_') <> 1
ORDER BY 1, 2"""


DEFAULTS_INFO_SQL = """SELECT table_name, column_name, data_default
FROM   user_tab_columns
WHERE  default_length IS NOT NULL
AND INSTR(table_name, 'X_') <> 1
ORDER BY table_name, column_name"""


VIEWS_INFO_SQL = """SELECT view_name, text
FROM user_views
ORDER BY view_name"""


TRIGGERS_INFO_SQL = """SELECT trigger_name, trigger_type, triggering_event, table_name, description, trigger_body
FROM user_triggers
WHERE INSTR(table_name, 'X_') <> 1
ORDER BY table_name, trigger_name"""


_CONN = None

def init_db_conn(username, passwd, tnsentry):
	global _CONN
	if not _CONN:
		dbinfo = username
		try:
			if USE_JYTHON:
				print(dbinfo)
				_CONN = zxJDBC.connect(username, passwd, tnsentry, 'oracle.jdbc.driver.OracleDriver')
			else:
				dbinfo = 'db: %s@%s' % (username, tnsentry)
				print(dbinfo)
				_CONN = cx_Oracle.connect(username, passwd, tnsentry)
		except:
			ex = sys.exc_info()
			s = 'Exception: %s: %s\n%s' % (ex[0], ex[1], dbinfo)
			print(s)
			return None
	return _CONN


def db_conn():
	return _CONN


def output_line(line):
	try:
		line = line.rstrip()
		for enc in OUT_ENCODINGS:
			try:
				line = line.encode(enc)
				break
			except UnicodeDecodeError:
				pass
		print(line)
	except UnicodeDecodeError:
		print('!!! line cannot be encoded !!!')
		pass


def select_qry(querystr):
	cur = db_conn().cursor()
	cur.execute(querystr)
	results = cur.fetchall()
	cur.close()
	return results


def run_qry(querystr):
	cur = db_conn().cursor()
	cur.execute(querystr)
	cur.close()


def fld2str(fld_v):
	if type(fld_v) == type(1.1):
		fld_v = '%s' % fld_v
		if '.' in fld_v:
			fld_v = fld_v.rstrip('0')
			fld_v = fld_v.rstrip('.')
	else:
		fld_v = '%s' % fld_v
		if fld_v.startswith('SYS_C'):
			fld_v = 'SYS_Cxxx'
	return fld_v


def show_qry(title, querystr, fld_join = '\t', row_separator = None):
	rs = select_qry(querystr)
	if rs:
		print('\n\n--- %s ---' % (title))
		for row in rs:
			line = fld_join.join([fld2str(s) for s in row])
			output_line(line)
			if row_separator:
				print(row_separator)


def init_session():
	run_qry("ALTER SESSION SET nls_numeric_characters = '.,'")


def show_tables():
	show_qry('tables', TABLE_NAMES_SQL)
	show_qry('columns', TABLE_INFO_SQL)


def show_primary_keys():
	show_qry('primary keys', PRIMARY_KEYS_INFO_SQL)


def show_indexes():
	show_qry('indexes', INDEXES_INFO_SQL)
	show_qry('indexes columns', INDEXES_COLUMNS_INFO_SQL)
	show_qry('composite indexes', COMPOSITE_INDEXES_COLUMNS_INFO_SQL)


def show_foreign_keys():
	show_qry('foreign keys', FOREIGN_KEYS_INFO_SQL)


def show_defaults():
	show_qry('defaults', DEFAULTS_INFO_SQL)


def show_views():
	show_qry('views', VIEWS_INFO_SQL, '\n', '\n\n')


def show_procedures():
	print('\n\n --- procedures ---')
	cur = db_conn().cursor()
	cur.execute('SELECT object_name FROM user_procedures ORDER BY 1')
	rows = cur.fetchall()
	for funname in rows:
		print('\n\n -- >>> %s >>> --' % (funname))
		cur.execute("SELECT text FROM user_source where name = '%s' ORDER BY line" % funname)
		lines = cur.fetchall()
		for line in lines:
			output_line(line[0])
		print('\n\n -- <<< %s <<< --' % (funname))
	cur.close()


def show_triggers():
	show_qry('triggers', TRIGGERS_INFO_SQL, '\n', '\n\n')


def test():
	main('user', 'passwd', 'tns')


def main(username, passwd, tnsentry):
	if not init_db_conn(username, passwd, tnsentry):
		print('Something is terribly wrong with db connection')
	else:
		init_session()
		show_tables()
		show_primary_keys()
		show_indexes()
		show_foreign_keys()
		show_defaults()
		show_views()
		show_triggers()
		show_procedures()
		print('\n\n--- the end ---')


if '--version' in sys.argv:
	print(__version__)
elif '--test' in sys.argv:
	test()
elif __name__ == '__main__':
	if len(sys.argv) != 4:
		print(USAGE)
	else:
		main(*sys.argv[1:])

Comments

  1. 1. At 7:01 p.m. on 10 oct 2008, David Lambert said:

    I wouldn't compare databases by converting them to text files followed by

    $ cmp db1.txt db2.txt
    

    data bases can get large. Suppose you collect data from three glass melting tanks for 10 years at sixtieth hertz. Each tank has 30 sensors.

    Otherwise, it looks like you hacked on the option parsing. Learn to use the optparse module. The command

    schema_ora.py username password
    

    fails with IndexError.

    I'd replace line 14 with a doc string, change line 15 to

    USE = __doc__
    

    and change lines 190--194 with

    elif __name__ == '__main__':
        if len(sys.argv) != 4:
        print USE
    else:
        main(*sys.argv[1:])
    

    Now, the database interactions look good to me, unless creating cursors is a long operation. If so, then rewrite to use one cursor. Maybe contexts are useful? In python 3.0, or earlier using "from __future__ import with_statement" you could write

    import contextlib #import contextmanager
    
    @contextlib.contextmanager
    def cursing(db_conn):
        cursor = db_conn.cursor()
        try:
            yield cursor
        finally:
            cursor.close()
    
    ...
    
    with cursing(db_conn) as cur:
        ...
    
  2. 2. At 8:33 a.m. on 13 oct 2008, Michal Niklas (the author) said:

    Thanks David,

    I do not want to compare database. All I want to know is if client db schema is identical with my. This tool is usable at preparing "patches" to change databas schema. Comparing data in database is more complicated. Especially with artifical primary keys created by database. Even with schema you will have "sys*" indexes that will be have diffent name on other database. I think other usable tool is dumping dictionary entries from database, so you can compare:

    languages table:
    -
    lang | short_id
    -
    English | en
    Polish | pl
    

    Dumping regular data to text is usable if you want to port data from one database (eg. Oracle) to other (eg. PostgreSQL) and there is no native way to do such import.

    I changed options parsing -- of course there was a bug with indexing.

    I know optparse but I think it is more usable in more complicated command lines.

    Regards, Michal

  3. 3. At 2:37 a.m. on 6 nov 2008, Michal Niklas (the author) said:

    I added similar recipe to dump PostgreSQL schema: http://code.activestate.com/recipes/576557/

  4. 4. At 2:32 a.m. on 19 jan 2009, Michal Niklas (the author) said:

    I added similar recipe for Informix: http://code.activestate.com/recipes/576621/

  5. 5. At 9:57 p.m. on 20 jan 2009, Michal Niklas (the author) said:

    2009-01-20: report from views

  6. 6. At 4:51 a.m. on 13 oct 2009, Michal Niklas (the author) said:

    Added support for Jython so recipe can use JDBC driver instead of cx_Oracle

  7. 7. At 8:14 a.m. on 21 oct 2009, foo bar said:

    Michal, nice script! Just what I needed (database source control has been ignored for a project I am working on). I used the output of the script and WinMerge to identify some schema inconsistencies.

Sign in to comment