reddit / emacs

Don't explain, show me examples! A tour of the catch/throw pattern in the Emacs source code...

tonyaldon · 2022-04-13 16:19
(This post in Org format -> [https://github.com/tonyaldon/posts](https://github.com/tonyaldon/posts)) Hey Elispers, Do you want to expand your Elisp toolbox? In this post we look at the `catch/throw` pattern offered by Elisp that allows to do nonlocal exits with the function `throw` that can be caught by the `catch` special form. For instance, in the following snippet, in a `catch` block: 1. we define a local list `l`, 2. then we loop forever (`(while t ...)`), 3. in this loop we generate a random (integer) number between `0` and `9`, 4. then: * if this number is not equal to `1`, we add it to the list `l` and we repeat, * and if it is equal to `1`, the `throw` statement transfers the control to the enclosing `catch` with the tag `:one` (we leave out the `while` loop and also the `let` block) which returns immediately the builded list `l` (last argument of the `throw` statement). Handmade examples are effective for discovering new things or remembering the syntax of known things. But when we have to write programs that solve "real" problems, having already been exposed to REAL WORLD examples is a competitive advantage. Indeed, REAL WORLD examples often provide "standard methods" to implement specific actions/tasks in given "environments". In this post, we first present some handmade examples of the `catch/throw` pattern and then we look at REAL WORLD examples of the `catch/throw` pattern that we find in Emacs source code. Let's go! # The catch/throw pattern: handmade examples In the info node about `catch` and `throw` (`M-x eval-expression RET (info "(elisp)Catch and Throw")`), we can read: Most control constructs affect only the flow of control within the construct itself. The function ‘throw’ is the exception to this rule of normal program execution: it performs a nonlocal exit on request. (There are other exceptions, but they are for error handling only.) ‘throw’ is used inside a ‘catch’, and jumps back to that ‘catch’. So with `throw` inside `catch` we can modify the flow of control. Let's see how with the following examples. We don't provide any explanation hoping that the evaluations in comments show how the flow of control has been modified. Note that if you read this post inside Emacs with org-mode, you can hit `C-c '` (`org-edit-special` by default) in the source block to open a dedicated `emacs-lisp` buffer where you can modify and evaluate the examples the way you want as much as you need to be confident about `catch` and `throw`. (catch :foo 'evaluated (throw :foo (+ 2 2)) 'not-evaluated) ; 4 (catch :foo (let ((a-value (+ 3 3))) 'evaluated (throw :foo a-value) 'not-evaluated)) ; 6 (catch 'foo 'evaluated (throw 'foo 'from-throw) 'not-evaluated) ; from-throw (let ((tag-catch 'foo) (tag-throw 'foo)) (catch tag-catch 'evaluated (throw tag-throw 'from-throw) 'not-evaluated)) ; from-throw (catch 'foo 'evaluated-1 (when nil (throw 'foo 'from-throw)) 'evaluated-2) ; evaluated-2 ;; nested `catch' (catch 'foo 'evaluated-1 (catch 'bar 'evaluated-2 (throw 'foo 'from-throw) 'not-evaluated) 'not-evaluated) ; from-throw (catch 'foo 'evaluated-1 (catch 'bar 'evaluated-2 (throw 'bar 'from-throw) 'not-evaluated) 'evaluated-3) ; evaluated-3 ;; `throw' called from another function (let ((f-throw (lambda (x) (throw :foo x)))) (catch :foo (funcall f-throw :bar))) ; :bar ;; raise an error (catch 'foo 'evaluated (throw 'bar t) 'not-evaluated) ; error: (no-catch bar t) # The catch/throw pattern: real world examples There are more than 1000 `catch` blocks in Emacs source code. Let's pick some of them that seems to represent in some way the "common" usage of `catch` blocks. Almost all those `catch` blocks follow the same structure: 1. enter in a `catch` block, 2. loop (either by iterating on a structure or by "traversing" a buffer), 3. for each iteration test something and decide if iterate or `throw`, 4. if thrown in the loop, leave the `catch` block and return the value passed to the `throw` statement, if ended the loop normally, evaluate the last parts of the `catch` block and return the last one. ## With dolist Sometimes, we want to loop over a list and if some "conditions" are verified for an item, we want to return without looping over the rest of the list. This can be done in a `catch` block using `dolist` with a structure similar to this one: (catch :tag (dolist (...) ... (when some-condition-is-true (throw :tag 'some-value))) ...) We encounter this pattern in the function `emacs-lock--exit-locked-buffer` that returns the first exit-locked buffer found in the list of all live buffers `(buffer-list)`: (defun emacs-lock--exit-locked-buffer () "Return the first exit-locked buffer found." (save-current-buffer (catch :found (dolist (buffer (buffer-list)) (set-buffer buffer) (unless (or (emacs-lock--can-auto-unlock 'exit) (memq emacs-lock-mode '(nil kill))) (throw :found buffer))) nil))) We also encounter this pattern in the function `handle-delete-frame` that handles delete-frame events from the X server. This function looks for a "valid frame" (among other stuff being different from the frame where the X event occured) in the list of frames returned by `(frame-list)` in order to decide if it is safe to delete the frame where the X event occured with `delete-frame` or if it is better to call the function `save-buffers-kill-emacs`: (defun handle-delete-frame (event) "Handle delete-frame events from the X server." (interactive "e") (let* ((frame (posn-window (event-start event)))) (if (catch 'other-frame (dolist (frame-1 (frame-list)) ;; A valid "other" frame is visible, has its `delete-before' ;; parameter unset and is not a child frame. (when (and (not (eq frame-1 frame)) (frame-visible-p frame-1) (not (frame-parent frame-1)) (not (frame-parameter frame-1 'delete-before))) (throw 'other-frame t)))) (delete-frame frame t) ;; xxx says it is ok to ask questions before terminating. (save-buffers-kill-emacs)))) Note that `handle-delete-frame` is bound to the event `delete-frame` in the keymap `special-event-map`. Now, let's have a look on the function `newsticker--icon-read`. This function is defined in the file `lisp/net/newst-reader.el` part of the package `newsticker.el` which is from its description section: ... an "Atom aggregator", "RSS reader", "RSS aggregator", and "Feed Reader". We did not choose this function for the service it provides to the package `newsticker.el` but because it is an interesting example dealing with two types of "controlled" nonlocal exits: 1. a nonlocal exit generated by `throw` and handled by `catch` and, 2. a nonlocal exit due to an error that can occur in a function (specifically `create-image`) and handled by `condition-case`. This function can be seen as a direct application of the material in the info node (`M-x eval-expression RET (info "(elisp)Nonlocal Exits")`). The function `newsticker--icon-read` takes an icon name as input, try to create and return an Emacs image for that icon looking for the image from the disk in the user newsticker directory, and if it can't (because it doesn't exist or it fails at creating the corresponding image) it returns a default image provided by Emacs distribution: (defun newsticker--icon-read (feed-name-symbol) "Read the cached icon for FEED-NAME-SYMBOL from disk. Return the image." (catch 'icon (when (file-exists-p (newsticker--icons-dir)) (dolist (file (directory-files (newsticker--icons-dir) t (concat (symbol-name feed-name-symbol) "\\..*"))) (condition-case error-data (throw 'icon (create-image file (and (fboundp 'imagemagick-types) (imagemagick-types) 'imagemagick) nil :ascent 'center :max-width 16 :max-height 16)) (error (message "Error: cannot create icon for %s: %s" feed-name-symbol error-data))))) ;; Fallback: default icon. (find-image '((:type png :file "newsticker/rss-feed.png" :ascent center))))) Leaving out the details of this function, we can extract a simplified scheme, that says: 1. in a `catch` block, if the directory `dir` doesn't exist, return a default image, if it exists loop over all the files in the directory `dir`, 2. in the loop, for each file try to create an image using that file, if it fails, log the error in the message buffer, if it succeeds, throw to the `catch` for the tag `icon` and return the created image from the `catch`: ## With re-search-forward In Org related packages, a technique used to find something in the buffer is to: 1. search in the buffer some part that match some regexp (with `re-search-forward`), 2. when we find that part, extract the information available at point (specifically we get it with `org-element-at-point`), 3. check some conditions on the element we've parsed, 4. depending on the result of the previous check, we continue the search in the buffer or we `throw` and return some result. This technique can be done with some code similar to this snippet: (let ((case-fold-search t) (re ...)) (catch :tag (while (re-search-forward re nil t) (let ((element (org-element-at-point))) (when ... (throw :tag ...)))))) We encounter this pattern in the following functions `org-log-beginning`, `org-babel-ref-resolve` and `org-columns-get-format`. We reproduce below the source code of `org-babel-find-named-result` which also uses that technique but enclosed in a `save-excursion` that saves the point and current buffer, execute what's in the body and restores those things: (defun org-babel-find-named-result (name) "Find a named result. Return the location of the result named NAME in the current buffer or nil if no such result exists." (save-excursion (goto-char (point-min)) (let ((case-fold-search t) (re (format "^[ \t]*#\\+%s.*?:[ \t]*%s[ \t]*$" org-babel-results-keyword (regexp-quote name)))) (catch :found (while (re-search-forward re nil t) (let ((element (org-element-at-point))) (when (or (eq (org-element-type element) 'keyword) (< (point) (org-element-property :post-affiliated element))) (throw :found (line-beginning-position))))))))) The same technique is also used but backward using `re-search-backward` like in the function `org-refresh-category-properties`. WE ARE DONE!!!
I_am_BrokenCog · 2022-04-13 18:17
> inside Emacs with org-mode, you can hit C-c ' (org-edit-special by default) this didn't work for me.
tonyaldon · 2022-04-13 19:07
I should have been more specific. You must be (the cursor) inside the source block #+BEGIN_SRC emacs-lisp ... #+END_SRC to press `C-c '`.
deaddyfreddy · 2022-04-13 19:25
isn't using `throw` just for returning a value an antipattern?
tonyaldon · 2022-04-13 20:07
Thanks u/deaddyfreddy for the comment. I don't know. Could you be more specific? I would appreciate it very much if you could show me with one of the "real world" examples in the post: 1. where there is an anti-pattern and, 2. how you would implement it differently.
deaddyfreddy · 2022-04-13 20:37
The problem with exceptions is they should be used for exceptional cases, like - IO error, division by 0 etc, so it's not possible to continue the code execution. Using them as a way to return a value is an overhead. Besides that, it breaks code flow unpredictably. So, instead of (catch :tag (dolist (x '(1 2)) (when (cl-oddp x) (throw :tag 'some-value))) nil) I would write something like (seq-some (lambda (x) (when (cl-oddp x) 'some-value)) '(1 2)) It even translates from algorithms more naturally, speaking of `emacs-lock--exit-locked-buffer` which is, according to your own comment > returns the first exit-locked buffer found in the list of all live buffers (buffer-list) which is literally what `seq-some` (actually, an adaptation of `some` from Clojure through dash.el's `-some`) does (seq-some (lambda (buffer) ; return the first buffer (set-buffer buffer) ;; which is exit-locked (unless (or (emacs-lock--can-auto-unlock 'exit) (memq emacs-lock-mode '(nil kill)))) buffer) ;; in the list of all live buffers (buffer-list)) There's no need to introduce new entities (catch, throw, or even dolist) here. Just translate the logic directly.
tonyaldon · 2022-04-13 21:56
Thank you for the detailed answer. >Just translate the logic directly. You're right using something like `seq-some` in the case of finding the first element in the list that satisfyied some conditions expose directly the intent. I take note. I was curious to see how `seq-some`, `-some` (from `dash.el`) and `some` (from Clojure) are implemented. It happens that `seq-some` uses catch/throw behind the scene: (cl-defgeneric seq-some (pred sequence) "Return non-nil if PRED is satisfied for at least one element of SEQUENCE. If so, return the first non-nil value returned by PRED." (catch 'seq--break (seq-doseq (elt sequence) (let ((result (funcall pred elt))) (when result (throw 'seq--break result)))) nil)) The implementation of `-some` doesn't use catch/throw (the job is done by `--each-while`). And the implementation of `some` from Clojure is really concise: (defn some "..." {:added "1.0" :static true} [pred coll] (when-let [s (seq coll)] (or (pred (first s)) (recur pred (next s))))) Can you recommend me some book (or better some source code) to read about "code flow" and "exceptions"? Have a nice day
7890yuiop · 2022-04-13 23:44
No, I think you're confusing this with the syntax used by some other languages for raising and handling exceptions. In elisp those would be "errors" and this is not about signalling and handling errors (there are other constructs for doing that). `catch` and `throw` are for non-local exits and that definitely and explicitly includes returning values. Fundamentally this is the only way of doing this in elisp (the likes of `cl-return` are implemented in terms of `catch`/`throw`). Not an anti-pattern at all.
usaoc · 2022-04-14 03:01
The evilness of `catch` is that it is *dynamic*, i.e. a catcher established by it has *indefinite scope* and dynamic extent. This is awful both in terms of the performance and the *power*. For non-local exits, you really want something more limited like Common Lisp’s `BLOCK` (available as `cl-block` in Elisp), which has *lexical scope* and dynamic extent. See [*Common Lisp: The Language*](https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html) for a detailed discussion.
usaoc · 2022-04-14 03:20
Side note: MacLisp programmers once had it confused as well by abusing `ERR` and `ERRSET`. That was why they later added `CATCH` and `THROW`, which Elisp inherited. See Section 6.2.2.1 of [*The Evolution of Lisp*](https://doi.org/10.1145/155360.155373) for the details. Elisp also has a *condition system* inspired by Lisp Machine Lisp, like Common Lisp does, although the condition system in Elisp is much less powerful than Common Lisp’s one. That’s why Elisp uses `signal` for signaling errors, unlike other “modern languages.”
usaoc · 2022-04-14 03:35
`catch` and `throw` are more like escape continuations in some Scheme implementations than exceptions, although a catcher established by `catch` has indefinite scope instead of lexical scope (`BLOCK` and `RETURN-FROM` in Common Lisp are *exactly* like escape continuations). Whether higher-order functions or explicit control flows are better is totally up to the programmer, so no comment on that, but do note that function objects are more expensive than you think in Elisp unless native compiled.
tonyaldon · 2022-04-14 08:05
Thank you u/asaoc for the explanation and the link. I didn't know about `cl-block`.
tonyaldon · 2022-04-14 08:14
>function objects are more expensive than you think in Elisp unless native compiled. This is not the first time I read that "function calls" are "slow". I also read it in [Why is there no tail recursion optimization ... (stackoverflow)](https://stackoverflow.com/questions/38493904/why-is-there-no-tail-recursion-optimization-in-emacs-lisp-not-but-like-other-sc): >the relatively inefficient implementation of function calls has influenced ELisp style I have no idea why this is happening. Do you know why is that?
usaoc · 2022-04-14 08:39
Not that *function calls* are slow, but that *constructing function objects* is slow. If you’re using lexical binding, then function calls are relatively fast since no dynamic access happens unless a special variable is present. The overhead of function calls is almost none as compared to the overhead of constructing the function objects in such case. It doesn’t help that the Elisp interpreter and byte compiler only provide naive optimizations as compared to the native compiler. Chris has some excellent writings on [lexical binding](https://nullprogram.com/blog/2016/12/22/) (a minor nitpicking: “lexical scope” is a misnomer) and [optimizing Elisp code](https://nullprogram.com/blog/2017/01/30/). The Elisp manual has [a section on function objects](https://www.gnu.org/software/emacs/manual/html_node/elisp/What-Is-a-Function.html), which alludes to the fact that interpreted function objects are `cons`es, and byte-compiled function objects are vectors.
usaoc · 2022-04-14 09:05
I would add to this three minor points: 1. Common Lisp has higher-order functions, and Common Lisp programmers do use them since Common Lisp implementations are quite optimized in this regard. Clojure’s `some` is similar to Common Lisp’s `FIND-IF`, which is available as `cl-find-if` in Elisp. Nonetheless, Common Lisp programmers also make use of `BLOCK` and `LOOP` when those make sense. `LOOP` is available as `cl-loop` in Elisp, which provides a `thereis` clause that does the same thing. 2. `seq-*` functions are expensive. They are generic functions, and generic functions are dynamically dispatched in Elisp. Use `cl-*` functions if you want higher-order functions without an extra dependency. 3. Clojure’s `some` is implemented like this because Clojure has lazy sequences.
deaddyfreddy · 2022-04-14 09:09
> but do note that function objects are more expensive than you think in Elisp unless native compiled no need to worry about that unless the performance is not enough > Whether higher-order functions or explicit control flows are better is totally up to the programmer computers were made to make people's life easier, not vice versa, so I still prefer to write what I want to achieve, not how computer should do it
deaddyfreddy · 2022-04-14 09:16
> Fundamentally this is the only way of doing this in elisp 1. not really 2. that fact it's "fundamental" doesn't mean we have to do it manually and explicitly, after all, most modern CPUs built around jumps, but it doesn't mean we have to write in terms of GOTO
usaoc · 2022-04-14 09:18
I have nothing against any paradigm. The power of Lisp comes from the fact that Lisp allows one to write any paradigm and to extend the language as they see fit. The essence of Lisp is reasoning about the *program*, not a specific paradigm like “imperative” or “functional.” I was only arguing against the “anti-pattern” part, which was based on a misunderstanding of what `catch`-`throw` actually do in Elisp. The side note is for those who’d like to optimize every bit of their code! ;)
deaddyfreddy · 2022-04-14 09:24
> I have nothing against any paradigm. I didn't say a word about paradigms, actually. > The power of Lisp comes from the fact that Lisp allows one to write any paradigm and to extend the language as they see fit it's a weakness of (mostly old style) Lisps as well, people come with different background, and even when they think it fits - it doesn't mean it sits. > The side note is for those who’d like to optimize every bit of their code! I suppose those don't want to write in Elisp :)
7890yuiop · 2022-04-14 09:34
1. The documentation disagrees with you. 2. No idea what your point is, but no one is going to force you to use a language feature you don't want to use.
deaddyfreddy · 2022-04-14 09:44
> seq-* functions are expensive. They are generic functions, and generic functions are dynamically dispatched in Elisp. seq-some isn't THAT expensive, actually (benchmark-run 1000000 (catch :tag (dolist (x '(2 4 6 8 1)) (when (cl-oddp x) (throw :tag 'some-value))) nil)) (1.637641115 0 0.0) (benchmark-run 1000000 (seq-some (lambda (x) (when (cl-oddp x) 'some-value)) '(2 4 6 8 1))) (1.809606454 0 0.0) and, surprise! (benchmark-run 1000000 (-some (lambda (x) (when (cl-oddp x) 'some-value)) '(2 4 6 8 1))) (1.3336181010000001 0 0.0)
tonyaldon · 2022-04-14 09:46
Thank you :)
deaddyfreddy · 2022-04-14 10:00
> The documentation disagrees with you. I mean, you don't have to make a non-local exit if all you need is just to do iteration until some condition met > No idea what your point is my idea is `some`-likes get the "find first element which" across better, than `catch/throw` because they literally translate the task.
tonyaldon · 2022-04-14 10:00
Thank you for all those references :)
usaoc · 2022-04-14 10:01
Interesting. `-some` is actually implemented in terms of `while` loops, saving the extra catcher established in the `catch`-`throw` approach. Try the compiled version of `seq-some` though, and you’ll see how bad it is ;)
deaddyfreddy · 2022-04-14 10:54
> Try the compiled version of seq-some though, and you’ll see how bad it is about 2 times slower, not so bad anyway, and it's for a million runs. And `-some` still performs almost as good as `catch/throw`, while being more readable.