reddit / emacs
How do I dump state or similar?
Mysterious_Habit_547 · 2022-05-17 19:22
I use emacs to write documents, some documents get pretty unwidly 14k+ words which seems to hit some funny boundaries, when saving/undo takes 10-60 seconds but not on every save. I suspect this is either due to undo-tree or flyspell in conjunction with GC set to collect at 1gb, profiler gives me some information but the issue is very hard to reproduce.
Is there any way for me to 'save state' such that I can easily reload the same memory, so that I can better hunt down this issue?
Failing that my current plan is to cut chunks out of my emacs conf, and use xdotool to type junk data in buffers until I can reliably reproduce.
danderzei · 2022-05-17 20:04
I split my long documents and combine individual files with include clauses
alexmurray · 2022-05-17 23:44
1Gb of gc may take a long time to scan, try turning that down and see if it reduces the amount of time Emacs pauses.
Faerryn · 2022-05-18 00:43
I recommend you use `vundo` and `gcmh-mode` instead of using `undo-tree` and setting `gc-cons-threshold` manually,
7890yuiop · 2022-05-18 01:32
> in conjunction with GC set to collect at 1gb
This is madness. That's three orders of magnitude higher than recommended.
> "Madness? THIS. IS." \**...freeze for 60 seconds...*\* "BADLY CONFIGURED EMACS!" \**kicks the space where something had been a minute ago*\*
whhone · 2022-05-18 04:30
You can do this:
1) M-x toggle-debug-on-quit
2) when freezes, hit C-g
Then you will see the backtrace.
Mysterious_Habit_547 · 2022-05-18 10:28
Right, because an 800kb GC threshold is sane? I'm trying to track what's probably a memory leak, which is a straight up bug irrespective of my config. No need to be an asshole.
Edit: And your imitations are toxic as fuck.
Mysterious_Habit_547 · 2022-05-18 10:43
Out of interest, what do you mean by 'scan'? I'm not familiar with emacs internals nor any extensive understanding of GC, but surely it's searching for referenced data, which I assume would basically be a allocation table no more than a few kb?
Mysterious_Habit_547 · 2022-05-18 10:44
Ta, I'm open to this option, I actually thought undo-tree was emacs core now, but did search / couldn't find any good alternatives. gc-cons-threshold looks interesting also.
yantar92 · 2022-05-18 12:26
> profiler gives me some information but the issue is very hard to reproduce
Sharing the profiling report would be helpful for others to give some ideas. See M-x `profiler-report-write-profile`
7890yuiop · 2022-05-18 12:55
800K is sane enough for virtually every Emacs user in the world, so... yes. Which isn't to say you can't play with the settings but, at the risk of still seeming like an asshole, I stand by my original comment.
Or to put it another way: You're complaining about occasional long pauses and using a setting which I believe is guaranteed to produce occasional long pauses, and I'm suggesting that maybe that's your problem. Perhaps it will turn out that it's not the only problem, but it seems like an awfully good place to start.
alexmurray · 2022-05-18 13:21
So the Emacs garbage collector is a mark-and-sweep style one - [http://www.gnu.org/software/emacs/manual/html\_node/elisp/Garbage-Collection.html](http://www.gnu.org/software/emacs/manual/html_node/elisp/Garbage-Collection.html) \- and so it has to scan the memory of the allocated objects to find which ones are unused so it can then free them. If you set a really high gc-cons-threshold then Emacs will wait until it has allocated 1Gb of objects before it runs the gc - and with 1Gb of objects to check this will take a longer amount of time than if you use a smaller value. So there is a trade-off here - you can make it pause less often \*but\* pause for longer by using a high gc-cons-threshold - or you can make it pause more often but for less time by using a smaller gc-cons-threshold. So you just need to be careful by setting a large value here that you don't end up causing Emacs to pause for too long when it does finally get around to doing gc.
Mysterious_Habit_547 · 2022-05-18 19:23
I literally pointed to memory allocation as probably being at fault, and am asking for assistance in building a reproducible scenario. Not once did I complain, you sound like a complete ass.
7890yuiop · 2022-05-18 22:43
You did appear to have "a complaint" in the sense that something which shouldn't take a long time was taking a long time, but it's true that you haven't "complained" about it, so I apologise for my evidentially-offensive phrasing, and wish you good luck with whatever it is you're trying to figure out... I no longer think that I know what that is, which appears to have been my problem all along, but perhaps you should run Emacs in a VM and take regular snapshots of the VM state.
arthurno1 · 2022-05-19 10:26
14k+ words means nothing to Emacs, it is laughably small, unless you have each word be several thousands of characters and everything in on a single line of text ... or something else nuts, which you probably don't have.
You should first try with Emacs -Q. Open your document in pristine Emacs, without your setup, and try to open and write to your document. If it works faster, then it is your setup.
Also, it is a very good tip to leave GC settings alone, as you have been pointed out by other commenters.
schmooser · 2022-05-19 18:32
I had similar when saving my Emacs config which is Org Mode file of 10k+ lines of text. It became very annoying, digging deeper I found that I enabled unlimited undo, so Emacs has saved 10+ Gb of Elisp data structures with undo and was writing that file on every save. By removing this madness it became normal again.
Mysterious_Habit_547 · 2022-05-19 19:08
This is interesting, I noticed some undo-tree save files were 16mb+, I don't know when undo was saved to file, but I don't recall this being an issue until then. I also have flyspell on, which I think does some stuff on save also, so GC is my go-to, but I wouldn't be surprised if it was undo + flyspell causing havok also. Thanks.
Mysterious_Habit_547 · 2022-05-19 19:12
\> word be several thousands of characters and everything in on a single line of text
This is not the case, but also it shouldn't be an issue. Is there any work to fix this in emacs? There are times where I need to open minified files, nasty I know, but $dayjob requires it.
\> Should first try with Emacs -Q.
I'll be doing this, the problem is it's difficult to reproduce, I'll spin up a VM and try and reproduce with xdotool
\> Also, it is a very good tip to leave GC settings alone, as you have been pointed out by other commenters.
Oh I know, I suspect 1gb is the ultimate extreme and I'm not sure why I set it to that. I also think 800k is crazy small - I use daemon mode which probably adds to it. It sounds like it's well understood that GC is broken, I'm unsure if it \_needs\_ to be in the mainthread, it would be nice if it pre-processed in a subthread, then blocked main to revalidate + cleanup or something. But I can barely even read lisp, nor do I have time to learn, so I can't really help there.
arthurno1 · 2022-05-19 20:08
> > word be several thousands of characters and everything in on a single line of text
>
> This is not the case, but also it shouldn't be an issue.
Unfortunately, very wide lines are a limitation. Try to open a text buffer with several thousands of characters on the same line, and Emacs will struggle. A simple workaround is to either use an external tool like sed or Emacs itself (non-interactively) to insert new lines at strategic places and then open the file. Minified files usually have a not-minified sources available, but in case there is not, they probably have some syntax you can exploit to insert new lines as suggested.
> Is there any work to fix this in emacs?
No idea. Subscribe to [emacs-devel](https://lists.gnu.org/mailman/listinfo/emacs-devel) mailing list at gnu.org and ask there, or [search the archives](https://lists.gnu.org/archive/html/emacs-devel/).
> > Should first try with Emacs -Q.
> I'll be doing this, the problem is it's difficult to reproduce, I'll spin up a VM and try and reproduce with xdotool
You need neither a VM nor xdotool for this. Just open a terminal and run emacs -Q, and then open your file from within Emacs and see if time to save that file is shorter without your setup, or the same.
> I also think 800k is crazy small
No, it is not. It means GC will run more often for shorter durations. Usually while you read or edit your files, most of the time Emacs (or any other interactive process) is actually idling, since we humans are not that fast as computers. GC running more often for shorter periods of time will not be noticeable by humans, but GC running for longer period of time will be noticed in the form of the process appearing as "frozen".
> I use daemon mode which probably adds to it
No, it does not.
> It sounds like it's well understood that GC is broken
No GC in Emacs is not broken. It is just how simple mark-and-sweep algorithm works. It was the same in Java or other runtimes as well.
> I'm unsure if it _needs_ to be in the mainthread
No it does not *need*, but Emacs is inherently single threaded runtime, so GC in a separate thread would have to lock main thread anyway, so there is no win if it was in a separate thread. I am not an expert in garbage collection nor Emacs internals, but that is how I understand it. Maybe if it could block just objects per se, but considering that everything in Emacs is global state, it would probably be nightmarish to implement such GC reliably, without causing unintentional deadlocks and freezes.
> it would be nice if it pre-processed in a subthread, then blocked main to revalidate + cleanup or something
I don't know, as said, I am not an expert on that one, but if you are, I am sure a contribution will be appreciated.
> But I can barely even read lisp, nor do I have time to learn, so I can't really help there.
[That part is implemented in C](https://github.com/emacs-mirror/emacs/blob/master/src/alloc.c).