"Stackless" XEmacs?

[Glynn Clements]

Jerry James wrote:

> >   > 1. You have to choose between two painful alternatives.
> >   >    a. [...]
> >
> >  >    b. A scheduler that only runs inside of well-defined functions, meaning
> >   >       that a given stack can hog the CPU for arbitrary amounts of time.
> >
> >  It's not obvious to me that this wouldn't be a big win in XEmacs.
> >  This isn't a realtime OS where you need to discipline badly-behaved
> >  threads; it's a single-user application where judicious use of
> >  cooperative multithreading might permit big wins from a few tweaks.
> 
> Yes, but we already do that!  That's what the select()/poll() driven
> event loop is all about: find out what I/O is ready to go, then run
> the handler to drive it.  There are some places where we don't follow
> discipline and allow the process to block rather than defer activity
> to a later iteration of the event loop.

If your code is in the middle of a complex loop or recursion with a
lot of local state, it's a lot easier to call a hypthothetical "yield"
function than to turn your code inside-out so that it works as a
process filter.

The situation is analogous to converting a recursive-descent parser to
the pushdown automaton form created by e.g. yacc. The latter is a much
better fit for event-driven I/O, but who would write one by hand?

> >  Because threads are hard.  I could see using threads for certain
> >  subsystems (eg, redisplay, and assuming Marcus and Olivier are up for
> >  it, GC), but having true threads in the Lisp interpreter will be hard
> >  (unless we're willing to replace the whole thing).
> 
> Yes, threads are certainly hard.  I'm not necessarily suggesting we
> dive into using them.  I'm suggesting that the stackless approach is
> (a) a lot more work than fixing our discipline with respect to our
> current architecture for about the same benefit; and (b) a lot less
> work but with much less benefit than going to a threaded architecture.

AFAICT, the main issue is making dynamic scope and threads coexist. 
Having a separate stack for each thread doesn't help much when all of
your local variables are just pointers to global symbols.

-- 
Glynn Clements <glynn at gclements.plus.com>