Elisp: Ban Syntax Table

By Xah Lee. Date:

emacs syntax table is truely, really, greatly annoying. We need to write a parsing lib without syntax table.

every time i thought of using it and relying on it, i run into bunch of problems.

for example, recently i rewrote “xah-select-text-in-quote” so it's based on syntax table.

(defun xah-select-text-in-quote ()
  "Select text between ASCII quotes, single or double."
  (let (p1 p2)
    (if (nth 3 (syntax-ppss))
          (backward-up-list 1 "ESCAPE-STRINGS" "NO-SYNTAX-CROSSING")
          (setq p1 (point))
          (forward-sexp 1)
          (setq p2 (point))
          (goto-char (1+ p1))
          (set-mark (1- p2)))
        (error "Cursor not inside quote")))))

(note: backward-up-list changed in emacs 24.x. I'm using 24.4)

try the command in nxml-mode. Doesn't work. Why? probably because it has complex use of syntax table.

but syntax table is useful, no? No. It's not that emacs syntax table is useful. It's the underlying builtin emacs parsing lib that's useful.

For example, the select text in quote was like this, without using syntax table:

(defun select-text-in-quote ()
  "Select text between the nearest left and right delimiters.
Delimiters are paired characters: ()[]<>«»“”‘’「」, including \"\"."
  (let (b1 b2)
    (skip-chars-backward "^<>(“{[「«\"‘")
    (setq b1 (point))
    (skip-chars-forward "^<>)”}]」»\"’")
    (setq b2 (point))
    (set-mark b1)))

The problem was that, it couldn't deal with nested quotes or backslash escaped quotes. Nor can it deal with 'single quotes', as used in {Python, Ruby, HTML, …}. (If you include single quote as delimiter, it's a problem because single quote is also used as apostrophe, and happens often (For example, “it's so!”).) But at least, for any double quoted string that doesn't contain backslash escaped quotes, it always works, reliably.

so, the solution is to add perhaps 50 lines of code to do parsing, or, rely on emacs builtin parser. ((info "(elisp) Parsing Expressions"))

but if you rely on emacs parsing engine (such as the syntax-ppss function), it'll save you time writing the parser, but it relies on syntax table, meaning, your command's behavior is unpredictable, depending on each buffer/major-mode's syntax table.

Theoretically, the idea of syntax table is useful, because each major mode can have its own concept of quote, suitable for each language the major mode is designed for. Great. But in reality, it doesn't work out that way, as in this example of nxml-mode (which is written by the world's top xml expert James Clark, also a top emacs lisp expert. (who also wrote the classic html-mode and xml-mode in 1990s, all part of emacs.))

All these html modes had complex use of syntax table, because syntax table is not flexible. (For example, in HTML, the apostrophy isn't normally a quoting character, except when inside a tag (and not already inside a quote).)

<div class='wow' title="it's">complex</div>

Emacs's syntax table is mostly designed for just 3 languages of the 1990s: {C, Lisp, TeX}.


Emacs Modernization

  1. Simple Changes Emacs Should Adopt
  2. Why Emacs Keys are Painful
  3. Problems of the Scratch Buffer
  4. M-key Notation vs Alt+key Notation
  5. Menu Problem
  6. Mode Line Problem
  7. cua-mode Problems
  8. kill-buffer Induces Buffer Accumulation
  9. Emacs Form Feed ^L
  10. Inconsistency of Search Features
  11. Single Key to Delete Whole Line
  12. Emacs HTML Mode Sucks
  13. Emacs Does Not Support Viewing Images Files In Windows
  14. Emacs Spell Checker Problems
  15. Adopt HTML as Texinfo Replacement
  16. Support HTML Mail
  17. Problems of “man” Command
  18. Emacs Lisp Mode Syntax Coloring Problem
  19. Emacs Ahk Mode Problems
  20. Problems of Emacs's Manual
  21. Problems of Emacs's Manual; Examples
  22. Emacs: Have You Read Emacs Manual?
  23. Elisp: Ban Syntax Table
  24. Emacs: Make elisp-index-search use Current Symbol
  25. Emacs: Usability Problems of Mode Documentation
  26. Emacs GNU Texinfo Problems; Invalid HTML
  27. A Record of Frustration in IT Industry; Disappearing FSF URLs, 2006
  28. Emacs Manual Node Persistency Issues
  29. Emacs: dired-do-query-replace-regex Replace ALL (fixed)
  30. Problems of Emacs Supporting Obsolete Systems
  31. Elisp: Function to Copy/Delete a Dir Recursively (fixed)
  32. Thoughts on Common Lisp Scheme Lisp Based Emacs
  33. Text Editors Popularity and Market Research
  34. Text Editor's Cursor Movement Behavior (emacs, vi, Notepad++)
  35. GNU Emacs Development Inefficiency
  36. Emacs Dev Inefficiency and Emacs Web 2.0?
  37. Letter-Case Commands Usability Problems
  38. Emacs: Select Line, between Quotes, Extend Selection
  39. Emacs: isearch Current Word
  40. Suggestions on Line Wrap Commands
  41. Emacs: Single Key to Delete Whole Line
  42. Emacs Undo and Emacs Cult Problem

If you have a question, put $5 at patreon and message me.
Or Buy Xah Emacs Tutorial
Or buy a nice keyboard: Best Keyboards for Emacs


Emacs Lisp