"Stackless" XEmacs?

[Jerry James]

On Wed, Apr 30, 2008 at 12:04 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Jerry James writes:
>
>   > The word "stackless" appears to be a misnomer.  Any idea where that
>   > came from?
>
>  From a particular implementation of Python called "Stackless Python".
>  It's been around for many years, but Guido consistently refuses to put
>  it into CPython, I don't know why offhand.  It has a number of
>  supporters besides its author, though, and is well-debugged.

Interesting.  I'll look that up and see what they've done.

>   > 1. You have to choose between two painful alternatives.
>   >    a. [...]
>
>  >    b. A scheduler that only runs inside of well-defined functions, meaning
>   >       that a given stack can hog the CPU for arbitrary amounts of time.
>
>  It's not obvious to me that this wouldn't be a big win in XEmacs.
>  This isn't a realtime OS where you need to discipline badly-behaved
>  threads; it's a single-user application where judicious use of
>  cooperative multithreading might permit big wins from a few tweaks.

Yes, but we already do that!  That's what the select()/poll() driven
event loop is all about: find out what I/O is ready to go, then run
the handler to drive it.  There are some places where we don't follow
discipline and allow the process to block rather than defer activity
to a later iteration of the event loop.  Fixing those would give us
the same benefits as this "stackless" architecture.

In effect, we keep information on the heap that would be sitting on
separate stacks in the "stackless" approach.  We're more stackless
than the stackless approach!

>   > 3. There are several mature, well-debugged thread libraries out there.  Why
>   >    ignore them and start the development and debugging process all over
>   >    again?
>
>  Because threads are hard.  I could see using threads for certain
>  subsystems (eg, redisplay, and assuming Marcus and Olivier are up for
>  it, GC), but having true threads in the Lisp interpreter will be hard
>  (unless we're willing to replace the whole thing).

Yes, threads are certainly hard.  I'm not necessarily suggesting we
dive into using them.  I'm suggesting that the stackless approach is
(a) a lot more work than fixing our discipline with respect to our
current architecture for about the same benefit; and (b) a lot less
work but with much less benefit than going to a threaded architecture.

>   > 5. The world is going multi-core.  Intel is talking about having 80-core
>   >    processors available in 2 years.
>
>  Why wait for Intel?  You probably already have a couple lying around
>  (cf this month's ACM Queue---wow! an interview with Tom Duff and he's
>  not even the focus of attention? Yikes!)

I haven't remembered to read Queue for months, since I stopped getting
the print issue.  I'll have to dive online and read this issue.

>   > I am interested in having a discussion on what it would take to
>   > switch from our Reactor-based architecture to a multithreaded one.
>   > Such a switch would have serious implications for the Lisp engine
>   > and the garbage collector, for example.
>
>  Well, I have to admit that I will have very little useful to
>  contribute, but I'd be happy to watch and cheer from the sidelines!

I've got too many irons in the fire right now.  Let's postpone this
discussion for a few weeks.
-- 
Jerry James
http://loganjerry.googlepages.com/