setting start-process environment, too much GC, debugging XEmacs

[Nix]

On Mon, 23 Oct 2006, stephen at xemacs.org said:
> Nix suggests:
> 
>  > (start-process doesn't seem to support running programs with
>  > different environments yet, but that can't be too terribly hard to add,
>  > and is of general utility.)
> 
>     (let ((process-environment an-appropriately-constructed-list))
>       (start-process ...))
> 
> (Untested but should work.)  I prefer that idiom rather than
> cluttering up the start-process API even more than it already is.

I *always* forget `process-environment', and for some reason I never
remember to hyper-apropos for it... obviously that's the Right Way, yes;
there is no need for a change to `start-process' after all.

>  > (Why is XEmacs-21.5.27 choosing to garbage-collect between every word I
>  > type? If this is how the incremental GC normally works it's damned
>  > annoying. Yes, each GC round only takes about a second, but still, that's
>  > an unresponsive second every two seconds...)
> 
> Dunno.  I don't use the new GC yet.  File a bug report where Marcus
> will see it (M-x report-xemacs-bug).

Aside: the new GC, um, doesn't, despite running. After half an hour using Gnus:

  RSS    VSZ COMMAND
499000 789848 xemacs

`Eight hundred megabytes and constantly swapping.'

(phase 0.0
 n-gc-total 59.0
 n-cycles-total 344.0
 n-cycles-in-last-gc 1.0
 n-cycles-in-this-gc 1.0
 enqueued-total 55627565.0
 enqueued-in-last-gc 1104846.0
 enqueued-in-last-cycle 1104846.0
 enqueued-in-this-gc 1087451.0
 enqueued-in-this-cycle 1087451.0
 dequeued-total 55627565.0
 dequeued-in-last-gc 1104846.0
 dequeued-in-last-cycle 1104846.0
 dequeued-in-this-gc 1087451.0
 dequeued-in-this-cycle 1087451.0
 enqueued2-total 6552.0
 enqueued2-in-last-gc 0.0
 enqueued2-in-last-cycle 0.0
 enqueued2-in-this-gc 0.0
 enqueued2-in-this-cycle 0.0
 dequeued2-total 6552.0
 dequeued2-in-last-gc 0.0
 dequeued2-in-last-cycle 0.0
 dequeued2-in-this-gc 0.0
 dequeued2-in-this-cycle 0.0
 repushed-total 1021784.0
 repushed-in-last-gc 0.0
 repushed-in-last-cycle 0.0
 repushed-in-this-gc 0.0
 repushed-in-this-cycle 0.0
 finalized-total 1023642.0
 finalized-in-last-gc 8773.0
 finalized-in-last-cycle 8773.0
 finalized-in-this-gc 1726.0
 finalized-in-this-cycle 1726.0
 freed-total 130701690.0
 freed-in-last-gc 977110.0
 freed-in-last-cycle 977110.0
 freed-in-this-gc 1068832.0
 freed-in-this-cycle 1068832.0)

Note in particular that `freed-total' is less than an eighth of XEmacs's
memory image.


I think I'm going to restart XEmacs now. And then I'll file a bug
report.

(oops, too late, the oom-killer just zapped it. I love autosave.)

>  > One aside: if XEmacs goes into a tight loop, how do I debug it?
>  > I entered a newsgroup a few minutes ago and XEmacs wandered off
>  > computing madly, GCing occasionally, and never came back.

It did come back. It just took two hours.

I suspect this is an example of the `noticeable slowdowns' of
--with-assertions.

>  > The backtrace was, ahem, unhelpful:
> 
> Build with --with-union-type.  Source src/.gdbinit.  Type "lbt"

Aha! Lots of other nifty stuff in there too, thank you!

> The requirement for --with-union-type is an infelicity that appeared
> post-3.3 GCC; it used to be that GDB could get at the LISP_CHAR_TYPE
> macro, but for some reason it can't now.

Does building with -g3 help? (That includes macro information in the
debugging info, and makes that debugging info quite a lot larger in
the process.)

I suspect the GCC version number bump is because that's about when GCC
stopped using stabs debugging info by default on most ELF platforms and
switched to DWARF2. (You can still generate stabs debugging info with
-gstabs, and stabs debugging info with macros with -gstabs3, but there
is rarely a need for that unless you're cursed with old debuggers.)

> FWIW, there's known breakage in the regexp code that can lead to
> looping; it's claimed that simply incrementing the target text pointer
> under certain conditions where there is a null match DTRTs, but I've
> never been able to convince myself.

Well, in this case it wasn't a loop, just sloth. (And much of that sloth
may be because with XEmacs's memory image at that point hovering around
400Mb, L2 cache locality must have been absolutely awful. I'm not sure:
I'll try running it under cachegrind if it gets that slow again and see
what I can see.)

> Gnus (I assume you're using Gnus to read newsgroups) does a lot of
> weird stuff trying to make XEmacs appear to be GNU Emacs to its higher
> level code.  I find Gnus code unreadable, and it's rarely commented,
> so I've not even been able to formulate a bug report or RFE.

I've had quite a lot of experience picking Gnus code to bits (at least
enough to figure out how to advise its more gory functions: I have a
rather complex Gnus configuration). If it does turn out to be Gnus's
fault, I think I can send a bug report and/or patch to Lars.

But the same Gnus code worked fine with 21.4.19, so I suspect XEmacs
here. (Of course, it was just slow...)

-- 
`When we are born we have plenty of Hydrogen but as we age our
 Hydrogen pool becomes depleted.'