| Store | Cart

Re: [TCLCORE] TIP#308 Published: Tcl Database Connectivity (TDBC)

From: Kevin Kenny <kenn...@acm.org>
Fri, 16 Nov 2007 10:58:15 -0500
Donal K. Fellows wrote:
It occurs to me that I owe the community a cover letter presenting a
little bit of background, and assigning due credit to the sources of
various ideas.

Tcl's database access has for years been in a mild state of disarray.
The trouble is not that Tcl can't talk to databases.  In fact, Tcl has
an impressive array of database access codes available, and a good
many of them are quite easy to use and well maintained.  Nor is the
trouble that database access is not in the core language.  By and
large, database users recognize the need for a "database driver,"
supplied by the database vendor or a third party, to connect a given
database to a given interconnection fabric.  People install Oracle, or
MySQL, or Postgres95, or whatever, and expect that the installer will
provide an ODBC driver, a JDBC driver, or a Perl DBI (according to
their applications' programming environments) that will allow
connection.  Nobody expects the runtime environments of Java, Visual
Studio, or Perl to provide connectivity to all the databases.

What's missing, rather, is a certain sense that databases are
"supported by Tcl."  There's a certain feel that they are tolerated as
second class citizens rather than embraced as partners. There are
recurring questions, "doesn't Tcl have anything like JDBC (or DBI)?"
Clearly, something is needed to make databases feel more at home.

One cause of the feeling that databases aren't fully embraced, it
seems to me, is that their application programming interfaces (API's)
are so varied, since each was invented by the implementor of the
corresponding database interface module.  This variability gives Tcl's
database access the feel of being a collection of bits grafted onto
Tcl, rather than a unified whole. This is an area where, it appears,
only the Tcl Core Team can help.  While, for example, Michael Cleverly
has wrought a tremendous unification by implementing and releasing his
fine 'ns_tcl' package, it enjoys limited acceptance -- programmers
either do not hear of it, or else do not consider its benefits to be
worth the effort of installing yet another package, even one
implemented in pure Tcl.  Moreover, 'ns_tcl' places an unreasonable
burden on its maintainers, since they are the ones that are
responsible for grafting it atop the heterogeneous APIs of 'oratcl',
'mysqltcl', 'tclodbc', 'sqlite3', and so on.  The Tcl Core Team needs,
therefore, to use the bully pulpit that the office provides to
promulgate a uniform database API.

Building a uniform, "Tcl Core-supported," database API will actually
require comparatively little code in the Tcl Core itself.  Unlike most
of the other languages with database API's, Tcl enjoys a dynamic
nature that allows much of the "glue code" of a package like ODBC or
JDBC to vanish entirely.  If the interfaces associated with a database
are represented with command ensembles, then any ensemble that
implements the specified interfaces is _ipso facto_ a database
connection.  The body of code in ODBC and JDBC that is resposible for
relaying the programmer's request to the database driver therefore
vanishes; the driver simply talks directly to the program.

Even the "database open" primitive is left unspecified in TIP 308. The
reason, once again, is the dynamic nature of Tcl.  While JDBC, ODBC,
and the like generally provide a single "open" entry point, which
specifies a "connection string" to specify the database instance, Tcl
has no need of such a beast. Connection strings are
database-dependent, and the "open" operations of JDBC and ODBC simply
function as interpreters of the little language of connection strings,
allowing them to specify the driver and parameters. Tcl has a
perfectly good interpreter already; the "connection string" most
naturally appears as a Tcl command to evaluate.

Much of the rest of the design rationale will become apparent in
reading the TIP, and I won't say more about it in this
message. Instead, let me move on to the process we should follow in
fleshing out the TIP and considering it for acceptance.

This TIP has already been written with a good deal of informal support
from several maintainers of database interfaces, notably D. Richard
Hipp (sqlite3), Brett Schwarz (pgtcl), and Tom Poindexter (the
original author of oratcl).  If in limiting my consultations, I've
given the impression of giving short shrift to the others, I
apologize. At the stage of actually writing a first draft, I had the
distinct impression that it would suffer greatly from "design by
committee," and might easily devolve into an incoherent mess of
features. The time has now come for "many eyeballs" to examine the
draft, and I promise to make every effort to respond to the concerns
of the community.  In particular, I'm eager to hear from the
implementors of the many database APIs that are out there, telling me
what's ill-considered, what's unimplementable, what's insecure, ... in
general, what needs to be improved.

In fact, I've already identified a couple of areas that I plan to
rewrite in the next couple of days.  In particular, the section on
transactions needs to be reconsidered (I thank Donal Fellows for
pointing me to a better idea, which I'm now working on fleshing out);
and the [$statement execute] command, and the convenience procedures
that use it, needs to be reworked to allow for more graceful handling
of dynamically generated queries (Thanks are due to Joe English for a
concrete suggestion of how to fix this one).

A few aspects of the specification, however, I consider
non-negotiable. The representation of result set rows as dictionaries
(which I believed was unique to this spec, but have since discovered
was anticipated by pgtcl), the mandate of support for prepared
statements (needed to guard against SQL insertion attacks) and the
representation of database objects as instance commands are among

Moreover, I hope to be allowed to have the final say over the wording
of the specification as voted. I consider it essential to have a
single auctorial voice in such documents; I've seen too many of them
ruined by committees. I apologize for the presumption of arrogating
this role unto myself, and offer the weak defence that nobody else has
stepped up to do it.

I do commit not to call the vote until an implementation of the
specification, supporting at least one client-server and one
in-process database engine, is publicly available. I also commit to
withhold the vote while reasonable doubt remains that any of the
popular databases will be able to conform with the specification.
TIP 308 is therefore very much to be considered a work in progress for
some time yet.

73 de ke9tv/2, Kevin

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
Tcl-Core mailing list

Recent Messages in this Thread
Kevin Kenny Nov 16, 2007 03:58 pm