handling alloca overflow (Re: [Patch 21.4] native cygwin xemacs-21.4.19 segfaults when requiring un-define)

[Ben Wing]

Vin Shelton wrote:

>Comments?
>
>Does anyone think the below patch will have a noticeable effect on
>the speed byte compiler?  Barring any concerns, I will check this in for
>21.4.20.
>
>  - Vin
>
>  
>
BTW, 21.5 should be more or less immune to such problems; calls to 
alloca() generally go through some intermediate code (in ALLOCA() in 
lisp.h) that checks the size of allocation, and if it's over a certain 
amount, uses xemacs_c_alloca() instead. this does a regular C malloc(); 
cleanup is handled by recording the value of the stack pointer during 
each allocation and freeing everything that was previously allocated at 
a point higher up in the stack. to make this more effective i also 
worked in periodic cleanup calls in some common places -- every time we 
process an event (especially effective since this happens near the top 
of the stack), every time we allocate lisp data, and every tenth 
funcall. (another good place would be in a macroized version of 
check_what_happened(), so that it periodically gets hit by QUIT calls.)

steve, you might consider adding some docs about this to the internals 
manual; there's a large comment in 21.5 lisp.h around line 1100 that 
documents this in some detail.

OTOH, this error might still happen because the amount that triggers 
such evasive behavior is fairly large (256K bytes), and it only checks 
for single large allocations, not for multiple small ones that might 
collectively be too large. (however, if this is happening, the code is 
badly written; a function should absolutely *NOT* call alloca() inside 
of a potentially unbounded loop.)

aidan and/or steve, how can your stack be so small that a 160K 
allocation is hosing it? how much memory do you have in your machine, 
what OS?

also, 21.5 lisp.h has various other ways to explicitly do what the code 
below is trying to do; these all use an unwind-protect so they're 
guaranteed safe. they include MALLOC_OR_ALLOCA (which uses alloca() over 
a certain size, currently 64K), or xmalloc_and_record_unwind(), which 
just always allocates.

you might consider copying some of this code into 21.4 and using it.

ben

>Aidan Kehoe <kehoea at parhasard.net> writes:
>
>  
>
>> Ar an t-aonú lá is triochad de mí Iúil, scríobh Stephen J. Turnbull: 
>>
>> > The code at bytecode.c:1627 is the last line calling alloca_array
>> > below.  I wonder if maybe the compiler doesn't like the local
>> > declaration of struct jump in the middle of the declarations with
>> > initializations, or maybe either the compiler or cygwin.dll just
>> > doesn't handle alloca_array as we expect. 
>>
>>That’s it--it’s a mundane stack overflow. The length of the instructions
>>string is 79648, we try to allocate twice that (= 160k) on the stack,
>>Windows refuses. Mule-UCS is horrible code. 
>>
>>The below makes (require 'un-define) work for me. I’m not sure that it
>>doesn’t leak memory in the various error cases, unhappily. 
>>
>>    
>>
>===================================================================
>RCS file: /pack/xemacscvs/XEmacs/xemacs/src/bytecode.c,v
>retrieving revision 1.14.2.2
>diff -u -r1.14.2.2 bytecode.c
>--- bytecode.c	2005/01/31 02:55:04	1.14.2.2
>+++ bytecode.c	2006/07/31 10:36:05
>@@ -1624,7 +1624,7 @@
>     int from;
>     int to;
>   };
>-  struct jump * const jumps = alloca_array (struct jump, comfy_size);
>+  struct jump * const jumps = xnew_array (struct jump, comfy_size);
>   struct jump *jumps_ptr = jumps;
> 
>   Opbyte *program_ptr = program;
>@@ -1868,6 +1868,7 @@
> 
>   /* *program_ptr++ = 0; */
>   *program_length = program_ptr - program;
>+  xfree(jumps);
> }
> 
> /* Optimize the byte code and store the optimized program, only
>  
>
>>-- 
>>Santa Maradona, priez pour moi!
>>    
>>
>
>  
>