org-cc: Custom completions for Org (WIP)

[https://github.com/orgtre/org-cc](https://github.com/orgtre/org-cc) Hi, I started this project since I like the completion interface of [citar](https://github.com/emacs-citar/citar) so much. The basic idea is that it would be fantastic to have a citar-like interface for arbitrary structured subsets of Org entries. I) I started out trying to implement this using [marginalia](https://github.com/minad/marginalia), like the [consult ](https://github.com/minad/consult) commands, but quickly concluded that this wasn't the way to go here... please correct me if I'm wrong and there is more from these packages I could make use of. I also try to make use of as much of the citar codebase as possible, but have found it difficult so far: a lot seems too specific for bibliographic entries. II) At this point I still wonder if I'm not reinventing the wheel here and if there is not more of some existing package I could make use of? III) Speed is another concern, at least for larger use cases. In my example with 4000 entries the completion prompt takes 3 seconds to pop up. How difficult would it be to implement caching, hopefully using code from citar-cache.el or some other library? Is there another way to speed things up? A SQL database is overkill I presume? IV) Might there be a way to implement changing the sort order while the completing-read prompt is active? Or might it be a good idea to abandon completing-read completely for this and other features, like live editing? I am aware that some of this can be accomplished with [Embark](https://github.com/oantolin/embark). Org tables, org-columns and some other Emacs table libraries also go some way in this direction. The dream would be to have citar-like dynamic table construction + filtering + selection and Excel-like sorting + editing. Is anyone aware of any other package that goes into this direction? V) Finally, I wonder if there is some way to make the command specification cleaner? Nested alists are not so nice, but the alternatives (plists, structs, hashtables) seem worse. Any comments/suggestions/help would be much appreciated.

Font used?

In the org-mode buffer (upper part of pic): [Iosevka Aile](https://typeof.net/Iosevka/) 15. In the minibuffer (lower part of pic): Iosevka 15. For the star icon: [Font Awesome](https://fontawesome.com/search?q=star&o=r) via [all-the-icons.el](https://github.com/domtronn/all-the-icons.el). Theme: [doom-nord](https://github.com/doomemacs/themes). Additional styling (e.g. stars): [org-modern](https://github.com/minad/org-modern). Mode-line: [doom-modeline](https://github.com/seagle0128/doom-modeline). I also use the following: ```elisp (custom-set-faces '(fixed-pitch ((t (:family "Iosevka")))) '(variable-pitch ((t (:family "Iosevka Aile")))) '(org-drawer ((((class color) (min-colors 89)) (:weight normal :height 0.6 :foreground "gray85"))))) (add-hook 'org-mode-hook 'variable-pitch-mode) (add-hook 'org-mode-hook (lambda () (set-face-attribute 'org-table nil :inherit 'fixed-pitch))) ```

Thanks for the detailed info dude

II) org-ql. Kind of. But a good generic completion interface for Org is something that would be welcome. III) `org-map-entries` will be much faster on Org 9.6. You may keep relying on it. It is Org's job to optimize performance. IV) Have you seen Org column view (built-in)?

V) The canonical way is tagged plist. See https://www.gnu.org/software/emacs/manual/html_mono/cl.html#Argument-Lists

Have you tried `consult-org-heading`? I think expanding on that with a custom completion category then adding flexible keyword metadata via a marginalia category would be a quick route. It uses `org-map-entries` under the hood (which is indeed getting quite fast in recent org versions).

Thanks for your suggestion! Yes, this is exactly what I tried in the beginning (I). The main problem is a marginalia annotator acts independent from and after the construction of completion candidates. So one would have to call `org-map-entries` once to get all the headings, and then one would have to again navigate to each heading within the annotator function to get the other entry data. This is too inefficient I'd say. Moreover, I didn't want to assume that users actually always want the heading to display as a candidate, and I found it important to allow truncation of a field in the middle instead of only at the ending (especially for something like file paths or outline paths). But yeah, at a second look, there is probably more from consult/marginalia I could make use of, like `consult--read`.

Org column view goes in that direction yes! One would have to figure out a way to implement something like it across multiple files. Also it used to be unusably slow for most of my files, but I see that it is much faster now! Thanks a lot for your work on that and on making org-map-entries faster too! I kind of see what you mean with org-ql – I'll look into it more. How do you think a good generic completion interface for Org should look?

Thanks, I'll look into this.

> One would have to figure out a way to implement something like it across multiple files. You can use column view from agenda :) > How do you think a good generic completion interface for Org should look? I'd say that it should be customizable by user: 1. How the items are displayed 2. Grouping 3. Sorting 4. History Basically, things discussed in 21.6.7 Programmed Completion section of Elisp manual, but exposed to user and tailored for Org.

>You can use column view from agenda :) Wow, great! Edit: I gave a detailed example of how org-cc can basically be replaced by using columns-view from the org agenda [here](https://www.reddit.com/r/orgmode/comments/z9jrtq/comment/iynvs0z/?utm_source=share&utm_medium=web2x&context=3)...

Yeah that's a known issue. Sometimes hash maps are used behind the scenes to make this faster. You could also look into `org-element-cache-map`, which is blazingly fast and basically makes it like using a hash table to cache desired org entities and their properties; we had a discussion in citar about a similar mode using that (using the org-bibtex format in which bibtex data are translated into heading :PROPERTIES:). It scales to 10's of thousands of headings. /u/yantar92 probably knows more about the improvements happening there.

`org-map-entries` uses `org-element-cache-map` behind the scenes. Direct usage could be faster, but it will not allow tag matching features of `org-map-entries`.

One other idea is to forgo marginalia and roll your own annotation function, packaging the keywords/etc. as text properties in midflight, then extracting them when the annotation gets requested. [Here](https://github.com/minad/marginalia/issues/87#issuecomment-880724829) is minad demoing that technique for fish. You would save all the fancy formatting for the very last second since the annotation function is called lazily.

Thanks, I'll try it out.