so-long mode slow

Despite setting `(global-so-long-mode 1)` large files are still really unwhelding in my Emacs. I can't scroll at all and Emacs hangs, I have to frce-quit and reboot. Does anyone know how I can debug what's happening here?

> Despite setting (global-so-long-mode 1) The `so-long.el` is for *text* files with **long lines** (think of e.g. minified JavaScript or CSS libs). For **arbitrarily** large files (think GB) in terms of **size of the data** stored you may use e.g. [vlfi](https://github.com/m00natic/vlfi). As a fuzzy reference point, my Emacs handles 121.1 MB `du --all` output text in `text-mode` **without** `vlfi` pretty well(-ish). The same file open without Doom's bloat (`emacs -q`) works like a breeze. > Does anyone know how I can debug what's happening here? If you think it's Emacs **config** issue, then either - Run profiler - `M-x profiler-start` - `M-x profiler-report` - Bisect config Start without custom config at all emacs -q <file> If that works snappy enough, then gradually comment out text in your config file, until you pin a problem.

`global-so-long-mode` (assuming it actually triggered for the file in question. Did it?) should let you open files with super-long lines without *immediately* causing Emacs to be slow; but its abilities are limited. The documentation is pretty clear: > Note that while the measures taken can improve performance dramatically when > dealing with such files, this library does not have any effect on the > fundamental limitations of the Emacs redisplay code itself; and so if you do > need to edit the file, performance may still degrade as you get deeper into > the long lines. In such circumstances you may find that `longlines-mode' is > the most helpful facility. n.b. If your file is Very Large then longlines-mode will also take quite a while to process it.

Didn;t know about `vifi`, cheers!

Does vlfi (Version: 1.7) do something that vlf (Version: 1.7.2) does not?

Are you by any chance a Doom user? Doom does (or certainly used to do) some extremely ill-advised things with so-long. In particular it hacks things so that `so-long-minor-mode` keeps `font-lock-mode` on (when font-lock is frequently the biggest problem!), and it then arranges for that minor mode to be used in lots of cases, even if the major mode is still the default action. In my old copy of that repo, `core-editor.el` has all these hacks. Basically if you still have syntax highlighting enabled, that's going to be bad. See whether `M-x so-long-mode` in your buffer does anything different? It might not make a difference, and for certain kinds of file the performance is going to tank just as badly if you try to scroll very far through the buffer, regardless of anything; but if it happens to make a difference then it *might* help quite a lot.

I don't know what you're playing at. so-long was always a no-op for me. 2>/var/tmp/foo emacs-27.2 -Q -nw -l so-long \ --eval "(switch-to-buffer (url-retrieve-synchronously \"https://cdnjs.cloudflare.com/ajax/libs/plotly.js/2.8.3/plotly.min.js\" t))" \ --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) \ (forward-line 1))" \ --eval "(end-of-line)" \ --eval "(redisplay)" \ --eval "(let ((start (float-time))) \ (dotimes (_i 2) (save-excursion (scroll-down) \ (redisplay) (scroll-down) (redisplay))) \ (princ (format \"\nso-long--active[%S] %f seconds\n\" \ so-long--active (- (float-time) start)) \ (function external-debugging-output)))" \ --eval "(so-long-mode)" \ --eval "(let ((start (float-time))) \ (dotimes (_i 2) (save-excursion (scroll-down) \ (redisplay) (scroll-down) (redisplay))) \ (princ (format \"so-long--active[%S] %f seconds\n\" \ so-long--active (- (float-time) start)) \ (function external-debugging-output)))" \ --eval "(call-interactively (function save-buffers-kill-emacs))" && \ cat /var/tmp/foo Output: so-long--active[nil] 6.858532 seconds so-long--active[t] 6.878124 seconds

Yes, running redisplay at the end of some gigantic line is the *worst-case* of the thing that so-long can do nothing about at all (and you get progressively closer to that worst-case the further into the line you are). This is documented (and referred to elsewhere in this thread). The library is extremely useful in certain situations, and of very little use in others, but I think calling it "always a no-op" is silly (for example https://emacs.stackexchange.com/a/19030 demonstrates it eliminating a hang of 3.5 minutes).

>emacs.stackexchange.com/a/19030 demonstrates it eliminating a hang Must be nice having solutions looking for problems.

Here's a diff for you -- minor fixes cc02f25 * origin/master Fix typos 31b292d * fixed large-file-warning-threshold error a01e9ed * 1.7.1 Minor README additions. 25e16ef * Fix broken link in README df677c1 * Issue 35: make mode-line batch indicators more accurate b62bc04 * Issue 31 Make vlf-batch-size configurable in the remote case 55e0c40 * Explicitly set coding-system-for-read to the current coding when not inserting from the beginning. eaa3629 * Fix cursor position after search a8ba836 * Fix issue where tramp-verbose is bound to nil. 4eaf763 * 1.7 Rename vlf-integrate.el to vlf-setup.el

if you use counsel, you can use the action to open large files right from counsel-find-file (use-package vlf :defer t :ensure t :after (ivy counsel) :init (ivy-add-actions 'counsel-find-file '(("l" vlf "view large file"))))

> Must be nice having solutions looking for problems. It's also nice having tests which actually do what you imagined they were doing (or are in any way representative of typical real-world usage, at any rate). Try these variants instead: wget -O /tmp/plotly.min.js https://cdnjs.cloudflare.com/ajax/libs/plotly.js/2.8.3/plotly.min.js # find-file-literally emacs-27.2 -Q -nw --eval "(find-file-literally \"/tmp/plotly.min.js\")" --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) (forward-line 1))" --eval "(end-of-line)" --eval "(redisplay)" --eval "(let ((start (float-time))) (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) (scroll-down) (redisplay))) (princ (format \"find-file-literally %f seconds\n\" (- (float-time) start)) (function external-debugging-output)))" --eval "(kill-emacs)" \ 2>/tmp/output.literally # fundamental-mode emacs-27.2 -Q -nw --eval "(find-file \"/tmp/plotly.min.js\")" --eval "(fundamental-mode)" --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) (forward-line 1))" --eval "(end-of-line)" --eval "(redisplay)" --eval "(let ((start (float-time))) (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) (scroll-down) (redisplay))) (princ (format \"find-file fundamental-mode %f seconds\n\" (- (float-time) start)) (function external-debugging-output)))" --eval "(kill-emacs)" \ 2>/tmp/output.fundamental # find-file plus global-so-long-mode emacs-27.2 -Q -nw -f global-so-long-mode --eval "(find-file \"/tmp/plotly.min.js\")" --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) (forward-line 1))" --eval "(end-of-line)" --eval "(redisplay)" --eval "(let ((start (float-time))) (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) (scroll-down) (redisplay))) (princ (format \"so-long--active[%S] %f seconds\n\" so-long--active (- (float-time) start)) (function external-debugging-output)))" --eval "(kill-emacs)" \ 2>/tmp/output.so-long # find-file without global-so-long-mode emacs-27.2 -Q -nw -l so-long --eval "(find-file \"/tmp/plotly.min.js\")" --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) (forward-line 1))" --eval "(end-of-line)" --eval "(redisplay)" --eval "(let ((start (float-time))) (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) (scroll-down) (redisplay))) (princ (format \"so-long--active[%S] %f seconds\n\" so-long--active (- (float-time) start)) (function external-debugging-output)))" --eval "(kill-emacs)" \ 2>/tmp/output.default cat /tmp/output.literally /tmp/output.fundamental /tmp/output.so-long /tmp/output.default I get: find-file-literally 4.879263 seconds find-file fundamental-mode 28.982773 seconds so-long--active[t] 21.110686 seconds so-long--active[nil] 110.661358 seconds

You seem rather butt hurt about so-long-mode's defamation. Are you the author or something? Remind me what I get paying 5 times the time cost for so-long that I don't get with `find-file-literally`? If so-long-mode conveniently enables `bidi-inhibit-bpa` for you, then good job I guess? emacs-27.2 -Q -nw --eval "(find-file \"/tmp/plotly.min.js\")" \ --eval "(major-mode-suspend)" \ --eval "(setq bidi-inhibit-bpa t)" \ --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) \ (forward-line 1))" \ --eval "(end-of-line)" --eval "(redisplay)" \ --eval "(let ((start (float-time))) \ (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) \ (scroll-down) (redisplay))) \ (princ (format \"so-long--active[%S] %f seconds\n\" nil \ (- (float-time) start)) (function external-debugging-output)))" \ --eval "(kill-emacs)" \ 2>/tmp/output.so-bad emacs-27.2 -Q -nw -f global-so-long-mode \ --eval "(find-file \"/tmp/plotly.min.js\")" \ --eval "(while (< (- (line-end-position) (line-beginning-position)) 1.8e6) (forward-line 1))" \ --eval "(end-of-line)" --eval "(redisplay)" \ --eval "(let ((start (float-time))) \ (dotimes (_i 2) (save-excursion (scroll-down) (redisplay) \ (scroll-down) (redisplay))) \ (princ (format \"so-long--active[%S] %f seconds\n\" so-long--active \ (- (float-time) start)) (function external-debugging-output)))" \ --eval "(kill-emacs)" \ 2>/tmp/output.so-long Output: cat /tmp/output.so-bad /tmp/output.so-long so-long--active[nil] 24.906592 seconds so-long--active[t] 25.109945 seconds

So your argument appears to be: * so-long does some things * I too can do some things * ergo so-long is a no-op Which I would characterise as "missing the point". > Remind me what I get paying 5 times the time cost for so-long that I don't get with find-file-literally? That would be "everything that `find-file-literally` doesn't do". Obviously, I think. If `find-file-literally` is how you open all of your files all of the time, then more power to you. For other people, `so-long` is sometimes extremely useful. I'm sorry to have wasted my time by engaging in this apparently-pointless discussion.

FYI, Commercial Emacs no longer requires the user to remember to `find-file-literally` when opening problematic files. Commercial Emacs's performance on fragglerock's test: so-long--active[nil] 0.013767 seconds