Attachments getting wrong encoding when saved to disk [Was: font-lock-fontify-* / infinite loop]

[Aidan Kehoe]

 Ar an naoiú lá de mí Eanair, scríobh Aidan Kehoe: 

 >  > I see other strange things related to gnus. (Encoding problem #3) If I
 >  > in UTF8 buffer search for 'å' I search finds it if its there. (Key
 >  > sequence {C-så}) But in a UTF8 gnus summary buffer I get an "aring not
 >  > defined." message. What has gnus done to my keys?
 > 
 > I don’t know. I’ll try and reproduce this evening. 

Okay, I had the time to experiment a little further, and the issue is also
there with 21.4. Gnus initialises gnus-summary-mode-map as follows:

(defvar gnus-summary-mode-map
  (let ((keymap (make-keymap)))
    (gnus-suppress-keymap keymap)
    keymap))

gnus-suppress-keymap is a wrapper around suppress-keymap, which replaces all
the entries in a keymap bound to 'self-insert-command, with 'undefined. 
isearch sets the parent of isearch-mode-map to the bindings of the buffer at
hand. All the ASCII characters were bound to isearch-printing-char in
isearch-mode-map, but that wasn’t true for the non-ASCII characters, so the
keylookup hit the 'undefined answer for the buffer at hand, and gave up.

The below fixes that particular problem. Without it, should a conceptually
character keysym be bound to something other than self-insert-command in the
global map (say, every time you hit e you want some typing-break prompter to
increment a counter, and then call self-insert-command), typing that keysym
wouldn’t add to the search string, instead it’d break off the search (or hit
the suppressed definition in a parent keymap of isearch-mode-map as above)
The bit I’ve removed used to hard-code bindings for ASCII to work around
that, but only for ASCII, and that’s why North American folk weren’t seeing
the problem.

GNU added bindings for character classes using their char-table support in
keymaps, which seems to be a complete solution, as long as they don’t
introduce more character classes. Except, of course, their keymaps aren’t
opaque, so they’re fucking around with vectors instead of having an actual
API, and our X11-specifier keysyms. The below is also a complete solution,
in that it’ll search for every character typed. It relies on the interaction
of our just-in-time binding support with the
isearch-maybe-frob-keyboard-macros function, so it won’t be as reliable in
corner cases on 21.4.

Index: isearch-mode.el
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/lisp/isearch-mode.el,v
retrieving revision 1.15
diff -u -r1.15 isearch-mode.el
--- isearch-mode.el	2005/06/26 18:04:50	1.15
+++ isearch-mode.el	2007/01/09 18:26:47
@@ -239,17 +239,6 @@
   (let ((map (make-keymap)))
     (set-keymap-name map 'isearch-mode-map)
 
-    ;; Bind all printing characters to `isearch-printing-char'.
-    ;; This isn't normally necessary, but if a printing character were
-    ;; bound to something other than self-insert-command in global-map,
-    ;; then it would terminate the search and be executed without this.
-    (let ((i 32)
-	  (str (make-string 1 0)))
-      (while (< i 127)
-	(aset str 0 i)
-	(define-key map str 'isearch-printing-char)
-	(setq i (1+ i))))
-
     ;; Here FSF sets up various kludges to handle local bindings with
     ;; meta char prefix keys.  We don't need isearch-other-meta-char
     ;; because we handle things differently (via pre-command-hook).
@@ -278,6 +267,13 @@
 
     ;; Define keys for regexp chars * ? |.
     ;; Nothing special for + because it matches at least once.
+
+    ;; Note that the initialisation code for isearch-known-printable gets
+    ;; these characters at load time, not at run time--it requires that the
+    ;; init code for isearch-mode-map know what it's doing, and bind any
+    ;; characters that smell printable to isearch-printing-char explicitly,
+    ;; or to something else.
+
     (define-key map "*" 'isearch-*-char)
     (define-key map "?" 'isearch-*-char)
     (define-key map "|" 'isearch-|-char)
@@ -301,6 +297,35 @@
     map)
   "Keymap for isearch-mode.")
 
+(defsubst isearch-smells-printable (key &optional symbol-name)
+  "Does KEY appear to be a key normally bound to `self-insert-command'?
+
+Returns nil or the character it would be bound to.  Used to handle the case
+where a normally self-inserting key is bound to something else in the global
+map when calling isearch.  "
+  (when (symbolp key)
+    (setq symbol-name (symbol-name key))
+    (if (eq (length symbol-name) 1)
+        ;; For normal keysyms--the associated character is
+        ;; the first and only letter of the symbol name. 
+        (aref symbol-name 0)
+      ;; For X11 letterdiacritic or U1234 syntax keysyms, or control
+      ;; characters on other platforms.
+      (get-character-of-keysym key))))
+
+(defvar isearch-known-printable 
+  (let ((hash-table (make-hash-table :test 'eq :size 256)))
+    ;; This requires that the init code for isearch-mode-map know what it's
+    ;; doing, and bind any characters that smell printable to
+    ;; isearch-printing-char explicitly, or to something else.
+    (map-keymap (lambda (key binding) (if (isearch-smells-printable key)
+                                          (puthash key t hash-table)))
+                isearch-mode-map)
+    hash-table)
+  "Printable characters that isearch should treat as such.
+This overrides their possible suppression in parent keymaps with
+`suppress-keymap'.")
+
 ;; Some bindings you may want to put in your isearch-mode-hook.
 ;; Suggest some alternates...
 ;; (define-key isearch-mode-map "\C-t" 'isearch-toggle-case-fold)
@@ -554,9 +579,35 @@
 					 (and (current-local-map)
 					      (list (current-local-map)))))
  				 isearch-mode-map)
-	  isearch-selected-frame (selected-frame)
+	  isearch-selected-frame (selected-frame))
 
-	  )
+    ;; If there are any character keys in the global map that isearch
+    ;; doesn't know about, bind them to isearch-printing-char in the
+    ;; isearch-mode-map. This avoids their possible undefinition (with
+    ;; `suppress-keymap') in the current-minor-mode-maps and the
+    ;; current-local-map, particularly important for those non-ASCII X11
+    ;; keysyms that were given explicit bindings at startup.
+
+    ;; If the symbol smells like a character, give it a binding to
+    ;; isearch-printing-char as well. This handles the case where someone binds a
+    ;; letter to something other than self-insert-command at startup; we
+    ;; still want it to add to the string.
+
+    ;; Were it the case that we were sure that no code had called
+    ;; suppress-keymap since the last isearch call, we could omit this. 
+
+    (map-keymap
+     ;; symbol-name as an optional parameter because I don't want to do two
+     ;; sets of bindings. Yes, this is probably silly. 
+     (lambda (key binding &optional symbol-name) 
+       (when (and (null (gethash key isearch-known-printable))
+                  (or (eq 'self-insert-command binding)
+                      (isearch-smells-printable key)))
+	 (puthash key t isearch-known-printable)
+	 ;; Don't override 'space, and so on. 
+	 (if (null (lookup-key isearch-mode-map key))
+	     (define-key isearch-mode-map key 'isearch-printing-char))))
+     global-map)
 
     ;; XEmacs change: without clearing the match data, sometimes old values
     ;; of isearch-other-end get used.  Don't ask me why...


-- 
When I was in the scouts, the leader told me to pitch a tent. I couldn't
find any pitch, so I used creosote.