setting start-process environment, too much GC, debugging XEmacs

[Nix]

On Tue, 24 Oct 2006, stephen at xemacs.org mused:
> Nix writes:
> 
>  > I *always* forget `process-environment', and for some reason I never
>  > remember to hyper-apropos for it...
> 
> Well, it *should* be in the docstring for `start-process', for
> heaven's sake, but it ain't.  Doc fix forthcoming.  While I'm thinking
> about it, is there anything else that we omit reminding you about in
> process handling? :-)

Not that I know of. I mean, there's no need to remind me of my *last*
disastrous attempt to fix anything in that area (xemacs-21.4.7,
anyone?).

Of course there might be things I've forgotten about process handling
because they aren't documented: but I'm afraid I've forgotten about
them:/

>  > `Eight hundred megabytes and constantly swapping.'
> 
> Hrm.  Adrian reported this and I thought Marcus fixed it.  But that
> may be post-21.5.27.  When you say "21.5.27", do you mean tarball
> and/or cvs co -r r21.5.27,

That.

>                            or do you mean CVS HEAD and that's what
> `emacs-version' tells you?  Unless you're CVS-challenged for some
> reason, I strongly recommend using CVS HEAD---there's very little
> risk, as nobody is committing megapatches at the moment.  Stability is
> essentially monotonically increasing.

Oh, good. I'll upgrade this evening then. (The idea of an XEmacs trunk
existing for more than a week *without* Ben Wing submitting megapatches
is oddly disturbing.)

>  > I suspect this is an example of the `noticeable slowdowns' of
>  > --with-assertions.
> 
> That's possible.  Many algorithms go quadratic in buffer size because
> of error-checking on the position cache.  If the position cache is
> getting lots of hits, it will only be linear, but a lot of Gnus's
> algorithms, as well as fontlock which Gnus uses heavily, seem to loop
> over all buffer positions repeatedly.  That destroys locality, of
> course, and the position cache is useless.

It also maintains some unimaginably vast variables and loops over *them*
a lot, including checking their lengths frequently. (The group in
question has well over *30Mb* of overview data, so it tends to point
up slownesses in Gnus quite effectively. One of the reasons I've
upgraded to 21.5.x is for the way-better profiler I vaguely recall
noticing flying past on the list years ago, so I can try to stamp
on some low-hanging sloth fruit.)

> You could try configuring --with-error-checking=all,nobufpos.

On the list :)

>  > Does building with -g3 help? (That includes macro information in the
>  > debugging info, and makes that debugging info quite a lot larger in
>  > the process.)
> 
> You tell me.  (I'll try it in a few days, but the most real work I can
> do for the next couple is fixing docstrings. :-)

Docs are always good!

>  > I suspect the GCC version number bump is because that's about when GCC
>  > stopped using stabs debugging info by default on most ELF platforms and
>  > switched to DWARF2.
> 
> Ah, I noticed that but I didn't know what it meant.

stabs suck really quite notably. It's a terribly limited format which
can just about describe the behaviour of C-based programs without
breaking down and crying, and which can't even do that on some platforms
(OK, so Itanium isn't exactly a very widely-used platform, but once upon
a time people thought otherwise, and it triggered significant changes
in GCC before it became obvious that it was a white elephant with about
four serious users). Plus it's critical to C++ exception handling in
GCC 3.0 and above (guess why EH improved so fast? That's why).

DWARF2 is `less a debugging format, more a programming language' and can
seemingly describe just about anything. Among other things it allows all
live stack positions in all functions in the call stack to be identified
without the need for frame pointers, so those major platforms which
don't require FPs as part of the ABI and which are grossly slowed by
them (i386, I'm looking at you) can now disable them by default (and
will do as of GCC 4.2). (Downside: only very new versions of GDB can
*use* location lists: if a new GDB isn't released in time, the
-fomit-frame-pointer-by-default change will probably have to be reverted
and pushed back to GCC 4.3.)

Even the Linux kernel now provides backtraces by parsing DWARF2.

(Aside: the new GC is crashed by -fomit-frame-pointer. This may be
intrinsic to the new GC or it may be a codegen bug in GCC 4.1.1. I'll
look at it soon.)

The *downside* of DWARF2 is that it's much bigger unless you optimize
the redundancy out of it, which GCC is slowly starting to do because
there are people out there who generate enormous executables, and they
get annoyed when their 2Gb binary is supplemented by 500Gb of debugging
info! (-feliminate-dwarf2-dups -feliminate-unused-debug-types
-feliminate-unused-deubg-symbols turns this on, but don't try it before
GCC 4.2 and binutils 2.18 because there have been really rather nasty
bugs in the DWARF2 dup elimination that tend to break linking if you
turn it on... not every time, and the failure is obvious, but still.)

>  > (You can still generate stabs debugging info with
>  > -gstabs, and stabs debugging info with macros with -gstabs3, but there
>  > is rarely a need for that unless you're cursed with old debuggers.)
> 
> How old?  Mac OS X 10.3 uses a v5.3 gdb, and "cursed" doesn't begin
> to describe the way I feel half the time (and I don't even ever try to
> debug C++ :-( ).

DWARF2 support was added around 5.0 (but is not *fully* supported by any
release of GDB: but that's OK, because GCC doesn't emit all of DWARF2
yet, nor does anyone that I know of.)

>  > I've had quite a lot of experience picking Gnus code to bits (at least
>  > enough to figure out how to advise its more gory functions: I have a
>  > rather complex Gnus configuration). If it does turn out to be Gnus's
>  > fault, I think I can send a bug report and/or patch to Lars.
> 
> I don't think we can talk about "fault" here.  Gnus and XEmacs just
> don't think the same way AFAICT.

For things that don't think the same way, they get on quite well

>                                  But anything you can do to improve
> Gnus performance (or get our AUCTeX package up to current DAK version

That is also on my list. I dropped away from preview-latex development
a *long* time ago because I simply had too much else on my plate. I'm
no longer accepting that excuse from myself :))

> I'm not kidding about the AUCTeX package, by the way, but before
> anybody does significant work, notify at least me and Uwe Brauer,
> because we both have interest and experience along those lines.

OK.

-- 
`When we are born we have plenty of Hydrogen but as we age our
 Hydrogen pool becomes depleted.'