EDIT: (all the posts in Org format) [https://github.com/tonyaldon/posts](https://github.com/tonyaldon/posts)
Have you ever wondered how org-mode toggles the visibility of headings?
YES!!! me either!
Let's get into it ;)
Table of content:
* org-mode visibility of headings
* text properties
* Why are we talking about text properties if the mechanism of `outline-mode` uses overlays?
* overlays, invisible overlay property, buffer-invisibility-spec
* `buffer-invisibility-spec` equal to `t`
* `buffer-invisibility-spec` equal to `nil`
* `buffer-invisibility-spec` equal to `((foo) t)`
* ellipsis and `buffer-invisibility-spec` equal to `((foo . t) t)`
* default ellipsis `...`
* custom ellipsis modifying the display table
# org-mode visibility of headings
`org-mode` is built on top of `outline-mode` that is responsible of the visibility changes of the headings.
How does it work?
`outline-mode` uses overlays, specifically the overlay property `invisible` to toggle the visibility of the headings:
1. To hide the body of a heading, `outline-mode` makes an overlay from the end of the line of the heading to the end of the body of the heading, and set the property `invisible` of the overlay to be the symbol `outline`. Hence, the part of the buffer with this overlay is "replaced" (visually, not the content of the buffer) by ellipsis. Why is this? Because, when `outline-mode` is turned on, it adds the cons `(outline . t)` to the variable `buffer-invisibility-spec` which became buffer local and is responsible for the invisibility of each buffer.
2. To make visible the body of a heading, `outline-mode` removes any overlays in the body of the heading that have its property `invisible` set to the symbol `outline`.
To see exactly how this is achieved you can refer to the functions `outline-flag-region`, `outline-hide-entry` and `outline-show-entry` defined in the file `lisp/outline.el` and also the definition of the mode `outline-mode` in the same file.
You can get emacs's source code by running the following command:
git clone git://git.sv.gnu.org/emacs.git
OK!
The mechanism of `outline-mode` uses overlays, `buffer-invisibility-spec` and ellipsis.
But how do those "emacs/elisp features" play together?
In the next parts of this post, we build examples using them to try to get a good feel for their use.
# text properties
In a buffer, each character point can have text properties attached to it that can be used to do many things (like controling the appearance of the character).
For instance, in an `emacs-lisp-mode` buffer, with the following s-exp, and the cursor (the point) after the first parenthesis:
(setq my-var nil)
if we run:
M-x eval-expression RET (text-properties-at (point))
we get:
(face font-lock-keyword-face fontified t)
The character point "s" (point 2, i.e. the "s" at the second position in the buffer) has:
1. the text property `face` equal to the face `font-lock-keyword-face` which is why it is display with a different foreground color (depending on your theme) than the text `my-var` for instance,
2. the text property `fontified` equal to `t` which we don't describe here.
We can read more about the special text properties in the manual by running:
M-x eval-expression RET (info "(elisp)Special Properties")
If we want more information (not only the text properties) about the character point "s" (point 2), we can run (still with with the cursor after the first parenthesis):
M-x describe-char
which pops up the following help buffer:
position: 2 of 18 (6%), column: 1
character: s (displayed as s) (codepoint 115, #o163, #x73)
charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x73
script: latin
syntax: w which means: word
category: .:Base, L:Left-to-right (strong), a:ASCII, l:Latin, r:Roman
to input: type "C-x 8 RET 73" or "C-x 8 RET LATIN SMALL LETTER S"
buffer code: #x73
file code: #x73 (encoded by coding system prefer-utf-8-unix)
display: by this font (glyph code)
ftcrhb:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1 (#x56)
Character code properties: customize what to show
name: LATIN SMALL LETTER S
general-category: Ll (Letter, Lowercase)
decomposition: (115) ('s')
There are text properties here:
face font-lock-keyword-face
fontified t
Note that it might differs in your running emacs (different fonts, maybe information about overlays if you are using `hl-line-mode`, …).
# Why are we talking about text properties if the mechanism of outline-mode uses overlays?
Because:
1. Both text properties and overlays can "alter/control" the appearance of the buffer's text on the screen and so we have to know something important that is (from the manual `M-x eval-expression RET(info "(elisp)Overlay Properties"))`): all overlays take priority over text properties.
2. buffer invisibility can also be achieve with text property (for instance, this is what `org-mode` does to hide the brackets and the link part of links like this `[[link][description]]`), and it is important to notice it.
# overlays, invisible overlay property, buffer-invisibility-spec
We can make a part of a buffer invisible using:
1. the `invisible` text property (of that part),
2. the `invisible` overlay property ("on top of that part").
The "admitted" values of the `invisible` overlay property (or text property) and the invisibility effect expected depend on the value of the variable `buffer-invisibility-spec`.
In this section:
1. we define overlays,
2. we set the variable `buffer-invisibility-spec`,
3. we give different values to the `invisible` property of the overlays,
4. we observe the appearance of the buffer,
5. we repeat several times step 2) to 4).
6. we hope we get a good feeling of invisibility in Emacs.
Also note that all the evaluations of the s-expressions are done in the minibuffer with `M-x eval-expression` and the point in the buffer we operate on, that we call `*invisible*`.
Let's switch to the new "fresh" buffer `*invisible*` in `fundamental-mode` by evaluating the following s-exp:
(switch-to-buffer (get-buffer-create "*invisible*"))
Let's insert the characters `XXXXXX` at the beginning of the buffer `*invisible*`:
XXXXXX
## buffer-invisibility-spec equal to t
Now if we evaluate the variable `buffer-invisibility-spec`, we should get `t` (the default) in the echo area.
If not, we set this variable to `t` like this:
(setq buffer-invisibility-spec t)
Now, we make an overlay "on top" of `XXXXXX` (from point 1 to point 7 in the buffer) that we assign to the variable `ov-x` using `make-overlay`:
(setq ov-x (make-overlay 1 7))
and we see the following in the echo area:
#<overlay from 1 to 7 in *invisible*>
Now, by setting the property `invisible` of the overlay `ov-x` to `t` using the function `overlay-put` like this
(overlay-put ov-x 'invisible t)
we make the characters `XXXXXX` disappear.
This is due to the value of `buffer-invisibility-spec` equal to `t` (the default) which means that text is invisible if it has a non-nil `invisible` (text or overlay) property.
Now, evaluating the following s-exp sets `invisible` property of the overlay `ov-x` to `nil`
(overlay-put ov-x 'invisible nil)
makes the characters `XXXXXX` to reappear in the buffer `*invisible*`.
We also could have removed the overlay `ov-x` to make the characters `XXXXXX` to reappear. Let's see how.
First, as previously, we set the `invisible` property of the overlay `ov-x` to `t` to make the characters `XXXXXX` to disappear:
(overlay-put ov-x 'invisible t)
Then, instead of setting back the `invisible` overlay property to `nil` of `ov-x` we remove it. To do so, we use the function `remove-overlays` that let you remove all the overlays in a range of the buffer that have a specific property set to some value (in our case the property `invisible` set to `t` in the range 1 to 7 of the buffer).
So evaluating the following s-exp
(remove-overlays 1 7 'invisible t)
removes the overlay `ov-x` in the buffer `*invisible*` and make the characters `XXXXXX` to reappear.
## buffer-invisibility-spec equal to nil
As we removed the overlay `ov-x`, we redifined it as previously by evaluating the following s-exp:
(setq ov-x (make-overlay 1 7))
Let's set `buffer-invisibility-spec` to `nil`:
(setq buffer-invisibility-spec nil)
Then, by evaluating the following s-exp, we expect the characters `XXXXXX` to disappear:
(overlay-put ov-x 'invisible t)
BUT they don't.
This is normal, as we've just set `buffer-invisibility-spec` to `nil`, we've "disabled" the invisibility feature in the buffer `*invisible*`.
Now, we restore the `invisible` property of the overlay `ov-x` so as not to interfere with the next example by evaluating:
(overlay-put ov-x 'invisible nil)
## buffer-invisibility-spec equal to ((foo) t)
Let's add the characters `YYYYYY` after the characters `XXXXXX` with 3 dashes `---` in between such that the buffer `*invisible*` is now:
XXXXXX---YYYYYY
Now, we make an overlay "on top" of `YYYYYY` (from point 10 to point 16 in the buffer) that we assign to the variable `ov-y` using `make-overlay`:
(setq ov-y (make-overlay 10 16))
We set back `buffer-invisibility-spec` to `t` (the default):
(setq buffer-invisibility-spec t)
Then we add the list `(foo)` to the variable `buffer-invisibility-spec` using the function `add-to-invisibility-spec` as follow:
(add-to-invisibility-spec '(foo))
Now, the value of `buffer-invisibility-spec` is `((foo) t)`.
This implies that, now to make a part of the buffer invisible, the `invisible` property must be `foo` or `t`. Before, it could have been any value that is non-nil.
This way we can toggle the visibility of some parts of the buffer while other parts remain invisible (see `org-toggle-link-display` for instance).
Let's make `XXXXXX` disappear "permanently" by setting the `invisible` property of `ov-x` to `t`:
(overlay-put ov-x 'invisible t)
The characters `XXXXXX` disappear and the buffer `*invisible*` is now:
---YYYYYY
Now, we set the `invisible` property of `ov-y` to be equal to `foo`:
(overlay-put ov-y 'invisible 'foo)
The characters `YYYYYY` disappear and the buffer `*invisible*` is now:
---
Now, what we can do is to make `YYYYYY` appears again by removing `(foo)` from the invisibility spec `buffer-invisibility-spec` while the characters `XXXXXX` stay invisible:
(remove-from-invisibility-spec '(foo))
Now, the buffer `*invisible*` is:
---YYYYYY
Note that:
1. the overlay `ov-x` still has its property `invisible` equal to `t` and,
2. the overlay `ov-y` still has its property `invisible` equal to `foo`.
You can verify it by evaluating the following s-exp:
(overlay-get ov-x 'invisible) ; t
(overlay-get ov-y 'invisible) ; foo
# ellipsis and buffer-invisibility-spec equal to ((foo . t) t)
## default ellipsis ...
If the variable `buffer-invisibility-spec` as a list contains a cons `(foo . t)`, every continuous part of the buffer with the `invisible` property set to `foo` is replaced by ellipsis which are by default `...`.
The buffer `*invisible*` still contains the characters `XXXXXX---YYYYYY`, but maybe not all the characters are visible. So let's put our buffer in an appropriate state for this section.
We removes all the overlays in the buffer (which makes all the content of the buffer visible again). We redefined the `ov-x` and `ov-y` as previously (same part of the buffer (1 to 7) and (10 to 16)). And we set `buffer-invisibility-spec` to be `((foo . t) t)`. We can do this by evaluating the following expression (in the minibuffer with point in the buffer `*invisible*`):
(progn
(remove-overlays)
(setq ov-x (make-overlay 1 7))
(setq ov-y (make-overlay 10 16))
(setq buffer-invisibility-spec t)
(add-to-invisibility-spec '(foo . t)))
The buffer `*invisible*` is now:
XXXXXX---YYYYYY
By evaluating the following s-exp, we set the `invisible` property of the overlay `ov-y` to `foo`
(overlay-put ov-y 'invisible 'foo)
and this replaces (visually not the content of the buffer) the characters `YYYYYY` by the default ellipsis `...` and the buffer `*invisible*` looks like this:
XXXXXX---...
## custom ellipsis modifying the display table
We assume with the buffer `*invisible*` is in the same state as in the previous section.
Our goal in this section is to modify the default ellipsis `...`.
To do so we:
1. create a display table with the function `make-display-table`,
2. we set its special slot 4 (responsible of the display of the ellipsis) which must be a vector of glyph using the function `set-display-table-slot`,
3. we set the variable `buffer-display-table` of the buffer `*invisible*` to be this new display table,
4. we observe the appearance of the buffer `*invisible*`.
So by evaluating the following s-exp:
(let ((tbl (make-display-table))
(glyph-vector
(vector (make-glyph-code ?\ 'font-lock-warning-face)
(make-glyph-code ?\; 'font-lock-warning-face)
(make-glyph-code ?- 'font-lock-warning-face)
(make-glyph-code ?\) 'font-lock-warning-face))))
(set-display-table-slot tbl 4 glyph-vector)
(setq buffer-display-table tbl))
the buffer `*invisible*` should looks like this (if the `invisible` property of the overlay `ov-y` is still equal to `foo`):
XXXXXX--- ;-)
You can read more about character display and display table in the manual by running:
M-x eval-expression RET (info "(elisp)Character Display")
WE ARE DONE :-)