Async non-blocking JSONRPC (or lsp performance faster/comparable with other clients)

After spending endless hours doing perf optimizations in `lsp-mode` source code finally, I gave up on the efforts on achieving performance in the elisp layer because it is practically impossible. At some point, it became clear that even if the sequential execution of the requests is comparable with other editors the the fact that (almost)no IO work is performed when emacs UI thread is busy ultimately chokes both the server and the client(especially for the single-threaded servers). Here it Emacs fork https://github.com/emacs-lsp/emacs based latest(?) release 28.1 branch that uses separate thread(s) for processing json and communication with the language server thus taking more than 95% of the load off the main UI thread. In addition, it reduces the GC pressure because some of the objects never reach the lisp layer. Compile it just like a normal emacs(make sure to have --with-modules configuration flag) and it should work with the latest and greatest `lsp-mode`. The code is still not extensively tested and works only for Linux/Unix. With the help of https://github.com/606u we are going to add Windows support as well.

Wow, sounds very exciting!! BTW is it possible to develop it into more general solution, which will help other plugins?

In general yes. But you can achieve pretty much the same with Emacs modules without the need to change the core code. JSONRPC has to create a lot of elisp objects efficiently, thus being in a module will hurt the perf because there is overhead for creating elisp objects from the module.

Initially I thought about [telega.el](https://github.com/zevlg/telega.el), telegram client which is, as far as I know, also uses json to communicate with server part written with C

For things like that with minor refactorings, the code/approach can be reused.

Any plans to bring this to upstream Emacs? How much more work is there left to do?

Windows support is not there and I want to experiment with it a bit more(e. g. we can prevent canceled responses to go to the elisp layer). I will propose it for merging, if it is not accepted it will live on independently.

I'm guessing you already considered this, so just out of curiosity, would it be possible to do something similar using dynamic modules or are there limitations there that necessitate changes to the emacs core?

https://github.com/emacs-lsp/lsp-mode/issues/676

Do you have any benchmarks for Linux/unix? I can test this on macOS if it helps.

Practically, there is no difference when you do sequential calls because you have pretty much the same code running with JSONRPC having a bit more of the elisp part written in C. Sequential requests are already fast. The issue is that when there are a lot of request/responses and a lot of UI processing (e. g. rendering company popup, fontlock, etc) that have to be handled they will block each other.

Interesting, thanks. I’ll definitely give this a spin and see if I notice the difference on the big C++ codebase at work. On a semi-related note, thanks for all the work you (and other maintainers) have put into lsp-mode. It’s such a game changer for the ecosystem, I don’t think I could use emacs productively at work without it!

This is pretty cool. I wonder where the biggest win was, though. I always figured the current bottleneck is Lisp object allocation during parsing, but it's not currently parallelizable, in Lisp or native code. And libjansson itself was pretty fast in comparison. So does it come from reducing string copying (before they are handed to/from JSON parser/serializer) and Lisp function calls? Or it parallelizing the parsing itself really that beneficial?

>processing json and communication with the language server thus taking more than 95% of the load Do you think this also explains the advertised speed advantages of [lsp-bridge](https://github.com/manateelazycat/lsp-bridge)?

If build emacs29 with thread support, is that can help on improving performance?

3 and 1/2 big wins: 1. Jansson parsing was actually 95% of the time(I havent performed extensive measurements but that was the range). Back then when native json parsing was introduced it was 50/50 but then elisp allocation was optimized at some point by Eli. At that point, I was pretty much happy with what sequential performance tests were showing us but in reality that was not enough. It was still noticeable that emacs is slower than vscode for example. 2. Unblocking the server: we had a nice discussion with u/ericdallo several months ago. They(clojure-lsp) started performing the IO work on the thread processing the messages. And every other client was doing fine but Emacs. That was caused by the fact that the server is blocked waiting for emacs to read the input and it cannot process other messages... 3. Is the same as 2 but in the opposite direction: if the server is busy emacs is blocked on writing the jsonrpc message. No matter how fast the serialization it won't solve that issue. 4. GC load is reduced because we don't have to allocate lisp strings for the process communication. I haven't measured that.

I haven't tested it but I would imagine that the main reason for the better perf is reducing the IO load. At the same time, the async jsonrpc just handles the full IO load without affecting the UI thread perf.

Just to make sure we are on the same page: 1. It will improve only lsp-mode perf not generally emacs performance. 2. In order for that to work, you have to use the json-rpc branch from here: https://github.com/emacs-lsp/emacs .

Thanks for your comment, so there are two points 1. asyncing process the lsp request/response; 2. converting elisp<->json in native part Look forward the process on this. : )

I see, thanks. Eli improved things by about 2x by fixing string encoding conversion, but didn't do much (or anything?) to the time spend allocating Lisp structured. Maybe I was mistaken, and that part doesn't take too much time. Regarding #2 and #3, I wonder how much of an improvement it would be to just `release_global_lock; sys_thread_yield; ...; acquire_global_lock` around the `json_loads` and `json_dumps` calls in `json-parse-string` and `json-serialize`. We would still be slinging strings around, but libjansson's job could proceed in parallel to other communications. I think that kind of change could be the easiest to get into the core first.

> Eli improved things by about 2x by fixing string encoding conversion, but didn't do much (or anything?) to the time spend allocating Lisp structured. Maybe I was mistaken, and that part doesn't take too much time. Encoding is part of Jansson -> elisp conversion. > Regarding #2 and #3, I wonder how much of an improvement it would be to just release_global_lock; sys_thread_yield; ...; acquire_global_lock around the json_loads and json_dumps calls in json-parse-string and json-serialize. We would still be slinging strings around, but libjansson's job could proceed in parallel to other communications. > I think that kind of change could be the easiest to get into the core first. That's what I did initially. Then I moved the whole thing into a separate thread without realizing how much the improvement will be.

This doesn't seem to be using the builtin jsonrpc package, right? Isn't it already async and non blocking? Why is lsp-mode not using that? Is there something in that package that is lacking?

jsonrpc is blocking (also the jsonrpc implementation in lsp-mode). E. g. if the server is not ready to read the input emacs will be blocked. This is a fork of emacs that aims to eliminate that limitation and speed up the overall processing by offloading it from the UI thread.

Thanks for the response. Would this change also apply to the built-in jsonrpc implementation or is it separate? I'd hope that other emacs code using jsonrpc could also benefit.

It is separate. I guess one day if it is merged it will be supported by jsonrpc.el

Oh man this is so exciting. I get the occasional hang for completions and so on when a lot of stuff is happening in our big rust workspace. I will get this installed and play with it this week!

I definitely want to give this a try, hopefully this week. I've been working with Python/pyright quite a lot recently and sometimes there's glitches with completion just due to the number of symbols in third party libraries. 28.2 is the current 28.x version? I know Arch is currently running with 28.2. I have a 28.2 built under wsl/ubuntu which I'm working with. I had been running with a 29 latest build but I had issues publishing pdf/html papers using org-babel/python - a bug which may have been fixed, but there are some binary incompatibilities with compiled elisp which can confuse matters - especially when switching back down to lower versions.

I rebased on top of 28.2 but it should be easy to rebase on top of everything.

Oh wow - that is quick, I don't think I can go back now :-) So much smoother and quicker, it's instantly noticeable. Seriously great work - thank you very much !! I'll be using this as my main build for work, so can report any issues.

> Oh wow - that is quick, I don't think I can go back now :-) So much smoother and quicker, it's instantly noticeable. Yep. Just for fun for clangd I set lsp-idle-delay and eldoc-idle-delay to 0. Then I hold the up/down key and it is fast enough to update the highlights and eldoc info while the cursor moves. And I also run with a very fast cursor: > xset r rate 200 60 There are a few bugs I am aware of, but if the server is up and running and not crashing all the time you should be fine.

That sounds great! I couldn't get it to work on emacs master though – it says it can't start the lsp client `LSP :: TBD has exited ()`; for some haskell buffers it shows hints even though mode-line says `LSP[Disconnected]` and I end up with a whole bunch of `<defunct>` language servers. But perhaps these are bugs in the newest lsp-mode from git? I normally use the 8.0.0 release.

That happens from time to time on my side too. It then works if I try to start it from a different buffer. I will investigate it further

Hero! I've compiled "`lspemacs`" and have been running it for half a day. Results so far are very impressive. Faster and to me even more important: Emacs never freezes. My workspace currently has ~`3,400` java files, and once the Eclipse server has started and scanned the projects, it's snappy at all lsp operations I've tried so far.

Good to know. We will make it even faster. As a side note, are you aware of lsp-java-completion-max-results? It will be very useful for you.

Yes, indeed. I used to have it on `30`, but have cranked it up now that Emacs is speeding away on its newly found jet pack: ``` - lsp-java-completion-max-results 30 + lsp-java-completion-max-results 130 ```

I merged related changes to main branch locally on my machine, it compiles and runs, but sometimes memory usage goes to very high (up to more than 2GB), then emacs spends lots of time on GC and UI freeze... Not sure whether json-rpc is related to this behaviour... Have you guys ever seen this before?

I started running this fork today and have been blown away by the improvement, both for the TS language server and rust-analyzer. For some reason compiling vterm via my nix install is failing, but I’ll gladly trade vterm for the improved editing experience.

It is very unlikely that the changes in the fork broke vterm. I suspect it is caused by the fact that emacs was not configured with `--with-modules`. FWIW I am using vterm locally with the fork just fine.

Oh yeah for sure. It’s probably just that those compilation flags aren’t set up in the emacs overlay. Haven’t had time to dig into it, but I will soon

Seems like it was just bad timing and I ran into this: https://github.com/akermu/emacs-libvterm/issues/643

seems fixed by this MR: https://github.com/emacs-lsp/lsp-mode/pull/3823

Hey u/yyoncho ! Thank you very much for the amazing work! I have been using your fork for the last two days at work. I mostly use ts-lsp and eslint-lsp these days and I don't see any noticeable difference in terms of performance, but with your fork everything works so stable and less laggy. Before I needed to restart LSP every 10-30 minutes, but in the last 2 days I've only needed to restart LSP once.

I'd love to compile and try this out. Is there a recommended way to configure and build it? (E.g. recommended flags like --with-modules or --with-nativecomp etc.)

`--with-modules` will be needed. `--with-nativecomp` is up to you. `--with-json` will be also needed. It is on by default if jansson is present.

This sounds amazing, thanks for the great work! Can I ask a somewhat related question, but coming from a slightly different angle: would it help if one were to write a performant (read: C, rust etc) server whose job is to translate JSON into lisp objects in the format lsp expects and vice-versa? The idea is inspired on lsp-bridge \[1\], but with a twist - instead of caching etc, one would do a "dumb translation" between JSON and Lisp. I'm sure you guys probably discussed this sort of stuff before, so apologies if I'm rehashing old ideas, but it just occurred to me as I was reading this post and the docs on lsp-bridge :-) Thanks. \[1\] https://github.com/manateelazycat/lsp-bridge

Actually thinking about it, I suspect I am getting almost to where you are already :-) I was about to say, this wouldn't even need to be a server, it could be a C library that creates its own thread and parses the JSON - then I realised its started to sound a lot like the descriptions above :-) let me read more about your ideas, they are starting to sound even more amazing :-)

I have considered that in the past. The point is that there is no fast elisp object parsing and it won't matter if we are parsing from json or lisp string representation and we will still block the ui thread(due to the way process handling works).

> let me read more about your ideas It is not my idea - the way jsonrpc has to be implemented for UI applications is a no-brainer - you want all processing to happen outside of the UI thread and you just to get the data in the main thread. The challenge is how to fit that in emacs architecture. Native jsonrpc has almost everything that we want from a practical point of view except one thing that I will fix soon and IME it brings the "native" application feeling to lsp usage.

most likely fixed by https://github.com/emacs-lsp/emacs/pull/8

I tried applying that on the lsp/json-rpc-29 branch, but still get the same behaviour. Is it only tested to work on emacs 28, or should 29 work too?

The fixes were not cherry-picked in json-rpc-29.