[TCLCORE] Announce: Experimental NRE (non-recursive engine)

I have (finally!) gotten around to porting the NRE from the old 
mod-8-3-4 branch to HEAD. I am very pleased with the result, it is IMO 
much better than the old one, and has huge potential for adding new 
exciting capabilities to Tcl.


* WHAT *

What's this about NRE? In short, when a proc calls a proc we get a 
"sandwich" in the C stack: TEBC/TEOV/TEBC [*]. Recursion at the script 
level quickly consumes the C stack.

The NRE arranges for procs to execute in the same TEBC instance as the 
caller (assuming it was a bytecode already), no C stack consumption 
occurs. The Tcl stack is maintained as before, heap is still being 
consumed. It is essentially a trampoline implementation.

A new TEBC instance appears on the C stack only when a command is called 
that is not NRE aware. There is a relatively simple new API for 
extensions to arrange that their commands avoid this fate.

The version provided is incomplete but very usable: procs, lambdas, 
namespace imports and same-interp aliases are all NRE aware. Namespace 
ensembles not yet, but should be soon enough.

The whole thing is completely 8.5 compatible at the script level, and 
almost at the C api level WITH ONE EXCEPTION: extensions that call a 
command's objProc directly as
   (*objProc)(clientData, interp, objc, objv)
WILL BOMB if they happen to call an NRE aware command. They should 
instead do
   TclNR_CallObjProc(interp, objProc, clientData, objc, objv)
and all will be well.

As a sample of the new things that this enables, I have provided a 
::tcl::unsupported::tailcall implementation. A tailcall also clears the 
Tcl stack before running. It messes the error stacks and the level 
counts for uplevel, but in a completely deterministic way.

Another interesting thing: we could finally get rid of the nasty hacky 
unportable headachy stack checking mechanism in the core, as we will 
never get even near the stack limit. Look at the tests in tests/NRE.test 
for an example - increase the numbers as far as you want, it will run in 
whatever stack you have. Be careful - you will hate it if the tests 
consumes all RAM and your machine starts swapping.

* Where *

The thing is in a fossil repository at
    http://tcl.msofer.com:8080
(if you do not know what fossil is, there is a short explanation and 
links right there).

If you are interested, please do play with the baby. I welcome bug 
reports and improvement suggestions. I am not really setup to manage 
this properly as a project, but will if the need arises.

* License *

The usual Tcl license.


Cheers

Miguel Sofer

-----
[*] TEBC: TclExecuteByteCode(), the bytecode engine
     TEOV: Tcl_EvalObjv() (or TclEvalObjvInternal()), the main command
           evaluator

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Tcl-Core mailing list
Tcl-...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tcl-core