[ANN] org-ql: new rifle/smart predicate (works like org-rifle)

What does ANN stand for? When I searched, it just turned up stuff on neural networks (:

My guess is "announcement".

Hi Adam: I'm starting to look at your Org query packages but am a bit unclear. In Melpa, I only see helm-org-rifle and not org-rifle. Here I see you are merging features into org-ql, so does org-ql replace your prior rifle packages and I should just look at that. I'm most interested in a fast Org entry lookup that does not depend on helm and actually has the minimum dependencies. Thanks.

Hi Bob, Short version: just use `org-ql`. It's much faster and more powerful. Long version: `helm-org-rifle` was my first Org-searching package. It initially depended on Helm, but then I added some commands that don't require Helm. Since the name implied that Helm was absolutely required, I renamed the repository `org-rifle`, but I didn't get around to splitting the functionality into separate `org-rifle` and `helm-org-rifle` packages. By that time, I was already working on `org-ql`. Now, `org-ql` supersedes `helm-org-rifle`/`org-rifle`; it's much more powerful, flexible, and well-optimized. The only functional difference between them is a minor difference in how results are presented, which recent changes to `org-ql` narrow further. And that difference could be easily customized to be identical, if that's what a user wanted. The recently added `rifle`/`smart` predicate in `org-ql` mimics the way `org-rifle` matches search terms against entries, which seems very intuitive to me. So from my perspective, there's little-to-no reason to use `helm-org-rifle` anymore, but it still works fine, of course. Hope this helps. Let me know if you have more questions.

Perfect. Look forward to trying it. By the way, I put a .org suffix on my HyRolo contact file and was anle to test out helm-org-rifle right away. Thanks. Bob

> By the way, I put a .org suffix on my HyRolo contact file and was anle to test out helm-org-rifle right away. That's cool, the file formats must be similar enough for Org to use it as-is. :)

Hi Adam: I just installed the latest org-ql release on Emacs 27.1 and am trying to get the exact same effect I had with helm-org-rifle but I'm not seeing instantaneous per character searching (prompted for a full query string) and I'm getting an error when the search is initiated when trying to search across a single .org file exactly as I had done with helm-org-rifle. I do want the interactive helm-like narrowing; that is why I am trying the package. I am calling (org-ql-find "\~/.rolo.org") and getting this partial traceback: Debugger entered--Lisp error: (wrong-type-argument markerp nil) marker-buffer(nil) (set-buffer (marker-buffer marker)) (save-current-buffer (set-buffer (marker-buffer marker)) (goto-char marker) (display-buffer (current-buffer)) (run-hook-with-args 'org-ql-find-goto-hook)) (let\* ((completion-styles '(org-ql-find)) (completion-styles-alist (list (list 'org-ql-find --cl-try-- --cl-all-- "Org QL Find"))) (selected (completing-read prompt --cl-collection-- nil)) (marker (gethash selected table))) (save-current-buffer (set-buffer (marker-buffer marker)) (goto-char marker) (display-buffer (current-buffer)) (run-hook-with-args 'org-ql-find-goto-hook))) (let (--cl-action-- --cl-affix-- --cl-annotate-- --cl-snippet-- --cl-group-- --cl-try-- --cl-all-- --cl-collection--) (setq --cl-action-- #'(lambda nil (progn (font-lock-ensure (point-at-bol) (point-at-eol)) (let\* ((path (org-split-string (org-format-outline-path (org-get-outline-path t t) window-width nil "\\37") "\\37")) (path (if org-ql-find-reverse-paths (string-join (nreverse path) "\\\\") (string-join path "/")))) (puthash path (point-marker) table) path)))) (setq --cl-affix-- #'(lambda (completions) (let\* ((--cl-var-- completions) (completion nil) (marker nil) (todo-state nil) (snippet nil) (--cl-var-- nil)) (while (consp --cl-var--) (setq completion (car --cl-var--)) (setq marker (gethash completion table)) completion table) for todo-state = (if-let (it (org-entry-get marker "TODO")) (concat (propertize it 'face (org-get-todo-face it)) " ") "") for snippet = (if-let (it (snippet marker)) (propertize (concat " " it) 'face 'org-ql-find-snippet) "") collect (list completion todo-state snippet))) (annotate (candidate) (while-no-input (or (snippet (gethash candidate table)) ""))) (snippet (marker) (org-with-point-at marker (or (funcall org-ql-find-snippet-function snippet-regexp) (org-ql-find--snippet-simple)))) (group (candidate transform) (pcase transform (\`nil (buffer-name (marker-buffer (gethash candidate table)))) (\_ candidate))) (try (string \_table \_pred point &optional \_metadata) (cons string point)) (all (string table pred \_point) (all-completions string table pred)) (collection (str \_pred flag) (pcase flag ('metadata (list 'metadata (cons 'group-function #'group) (cons 'affixation-function #'affix) (cons 'annotation-function #'annotate))) (\`t (unless (string-empty-p str) (when query-filter (setf str (funcall query-filter str))) (pcase org-ql-find-snippet-function ('org-ql-find--snippet-regexp (setf query-tokens (--select (not (or (string-match-p (rx bos (1+ (not (any ":"))) ":") it) (< (length it) org-ql-find-snippet-minimum-token-length))) (split-string str nil t (rx space))) snippet-regexp (when query-tokens (rx-to-string \`(seq (optional (repeat 1 3 (repeat 1 15 (not space)) (0+ space))) bow (or ,@query-tokens) (0+ (not space)) (optional (repeat 1 3 (0+ space) (repeat 1 15 (not space)))))))))) (org-ql-select buffers-files (org-ql--query-string-to-sexp (concat query-prefix str)) :action #'action)))))) (unless (listp buffers-files) (setf buffers-files (list buffers-files))) (mapc #'org-ql--ensure-buffer buffers-files) (let\* ((completion-styles '(org-ql-find)) (completion-styles-alist (list (list 'org-ql-find #'try #'all "Org QL Find"))) (selected (completing-read prompt #'collection nil)) (marker (gethash selected table))) (with-current-buffer (marker-buffer marker) (goto-char marker) (display-buffer (current-buffer)) (run-hook-with-args 'org-ql-find-goto-hook)))))))) org-ql-find("\~/.rolo.org")

Hi Bob, I just tested on Emacs 28.1 in a clean config (i.e. only org-ql and its dependencies installed): ran `org-ql-find`, typed a word and pressed TAB, and it worked without error. Unfortunately, I can't test it on Emacs 27.1, so I can't tell if that's the problem. Are you using any completion packages like Vertico, or just default Emacs completion, or...? Thanks.

[deleted]

Yes, I tend to experiment with packages like vertigo et al. to understand the value. If you know of any incompatibilities maybe you could warn on package load. I can test tomorrow with Emacs 27.1 28.0.50 and 29.0.50. — Bob

As far as I know, `org-ql-find` works correctly with Vertico and with default Emacs completion. I've heard of issues with Selectrum, and it doesn't seem to work with `helm-mode`, but I don't know why; it seems like, if it works with default Emacs completion, it ought to work with other packages that are supposed to work with default Emacs completion APIs. I know that it works with Emacs 28.1. I don't intend to support unreleased Emacs versions per se, but if you find any new incompatibilities in 29.0, that could be a helpful early warning. Thanks.

I do have helm loaded, so that could be it or it could be the header lines at the front of HyRolo files. However, helm-org-rifle worked fine in this scenario. I’ll look into it when I can. For now, I can use the older package.

If you can, try using `org-ql-search` on that file with the same search query. If that works, that should narrow down the problem to the `org-ql-find` command rather than `org-ql`'s backend.

Yes, org-ql-search works. I took out the rolo file header, making it a normal org file and org-ql-find still fails so it is not the file header.

Please check this issue: https://github.com/alphapapa/org-ql/issues/284 It could be related to the problem.

I don’t use selectrum so I don’t see a connection. But it dies make me think to try it with a vanilla Emacs and report back.