Is there a chance TRAMP will be rearchitected at some point?

I strongly feel that remote code editing is one of the most important features for editors going forward. Tramp, while sometimes usable, has poor performance and I've found it to hang during many different operations. It's not easy to debug when things go wrong, and it seems as though it's just a product of old, single threaded code that does not compete with better ideas on remote code editing - a la VSCode's remote server. Will we ever see Emacs embrace an over the network client/server split and make remote code editing a first class feature like VSCode?

> Is there a chance TRAMP will be rearchitected at some point? No. But someone might make a completely different thing fulfulling the same need. > ...it seems as though it's just a product of old, single threaded code... Emacs is single-threaded, so anything that runs in Emacs is going to be, too.

so much this! but I think its too difficult to change TRAMP at its current form - we would need a library which does remote editing completely in client server setup. I think the ideal architecture would consist of a standalone CLI running in the server which exposes file operations through websockets (similar to VSCode), and emacs client service which calls those functions in async.

It doesn't need to be synchronous though, there are tramp operations that could be reworked to be async, no? I've thought about looking into this particularly for etags... Remote usage is pretty rough.

It's definitely been getting attention - check out the git history. 4 months ago there was a "Refactor TRAMP" commit, but in my most recent build there is still a lot of latency from TRAMP. It seems to do a whole load of round-trip checks for remote capabilities one at a time. When my ssh connections are via an AWS jumphost it's pretty painful - ~4-6s to e.g. open dired with a 120ms SSH round trip *after* I debugged to get rid of all extra activity like hunting for project roots. It's beyond my elisp skills and available time, I browsed around to see if I could see how to combine those round-trips into a single multi-check round-trip, but it wasn't obvious (unsurprising, or someone much more knowledgeable and skilled would have already done it). And I no longer spend much time with high-latency connections, and TRAMP works tolerably well for low-latency.

> Emacs is single-threaded, so anything that runs in Emacs is going to be, too. That's not as true as it used to be. Emacs 26 added some basic threading support, though it does not appear to be used much yet.

It's a language construct which can encode some programmer intent and make it easier to express certain kinds of code. However, the runtime is still single-threaded, so even if you're using `make-thread` in elisp, it's running cooperatively in a single thread, and if that thread wedges, so does Emacs. It's not true parallelism.

Darn, I didn’t realize it was that limited.

>we would need a library which does remote editing completely in client server setup Don't we already have something like this? Admittedly I haven't kept up to date with new Emacs features for many years and my Modern Emacs-fu is limited, but couldn't something like emacsclient fill this need? On the other hand, it depends what level of abstraction you want to take in the client-server relationship. SSH seems to be the most portable and widely known. Likewise we have X-forwarding. And a whole load of other things I can't even think of right now.

That threading implementation has (or had, at least) [some serious limitations](https://nullprogram.com/blog/2018/05/31/) on top of those u/ieure mentions.

Instead why not a child process that does the work similarly to [mu4e](https://www.djcbsoftware.nl/code/mu/mu4e.html) + [mu-server](https://github.com/djcb/mu/blob/master/lib/mu-server.cc)?

I think people here are already confused between native threads vs async threads. I hate that so many languages refer to everything as threads when they aren't threads at all. Rust calls native threads "threads" and calls async stuff "tasks" which I find a lot clearer. Even if Emacs is single threaded, it can be async and concurrent using tasks.

I hope the replacement will be named LADY

Could remote editing be delegated to a child process, as with language servers, that would implement parallelism on its own and communicate asynchronously with Emacs? Or is this a naive idea that is infeasible?

[removed]

emacsclient is a different thing. There's still only one Emacs process, but with emacsclient you have a lightweight program that can tell it to do things. That can be handy (you can set `EDITOR=emacsclient` for `git commit` and that kind of thing, using an existing Emacs rather than starting up a new one for each).

I think the comparison here is highly flawed because TRAMP uses ssh to copy files between your machine and the remote machine whereas VSCode's remote server means running a VS code instance on the remote machine that you remote control (IIRC) via your usual VSCode interface. The emacs version of this is possible. First on the remote machine (login via ssh, of course), run: `emacs --daemon.` This will create an emacs server on the remote machine, persistently. Then next time you login via ssh, run:`emacsclient -c server` This will open the emacs running on that server. You can do your work, use the server's resources, and if you ever leave, the state will be saved so long as the server doesn't kill anything and emacs doesn't crash. If your emacs supports x11 forwarding and your terminal does too, then running this should spawn an emacs gui on your machine that controls the remote emacs server. If you ever want to stop the server, run: `emacsclient -e '(kill-emacs)'`

yeah this is exactly what I had in mind. A mu style standalone server which does the heavy lifting on remote, and a local client which interacts with it.

Async tasks require threads to run, right? Even if it's the main thread and the runtime switches back and forth, that's still a thread.

Sure, but that's like saying many cars run on one road. Cars are very different from roads. What Tramp needs to solve the issue OP is talking about are cars / tasks. Everything runs on at least one thread, so using that term is just confusing.

I am objecting to your term "async threads." I don't know what that means.

It could be local though, it just needs to be able to use ssh, scp & whatever other transports you need implemented. It'll then reply to Emacs asynchronously. It's generally a better idea to put more demands on the client than the server for additional things to be installed. That's pretty much the whole idea behind TRAMP to start with. Convincing work/whatever to install Emacs on the servers can be an annoying argument to have. Convincing them to install ssh which they need anyway to administrate them? Much easier.

"async threads" is not a real term. I'm making the distinction between native threads and async [fill this in based on your language]. Some languages call them green threads. Some call them go routines. Some Promises. Some Tasks. Etc. The terminology various languages use for async is very messy.

> Async tasks require threads to run, right? Not in the way most people would understand the term "thread" to mean. > Even if it's the main thread and the runtime switches back and forth, that's still a thread. This is cooperative multitasking, and Emacs has supported it for ages. It sucks for the same reason it sucked in the 80s and 90s, which is that a single misbehaving or broken thing -- like code which is trying to save a modified buffer over a network which is unavailable or congested -- breaks all the other stuff trying to run concurrently.

Yeah but the main thread is still a thread, so at the very least, a task requires one thread.

> Could remote editing be delegated to a child process, as with language servers, that would implement parallelism on its own and communicate asynchronously with Emacs? It's not clear to me what parallelism in the child process would be used for. Your suggestion sounds like a more complicated version of what TRAMP is already doing -- it spawns a process (ssh into remote host -> run bash), issues it commands (shell commands) and reads its output asynchronously via a pipe.

Oohh you could pipe from one to the other, through some sort of noodle…

> Yeah but the main thread is still a thread, so at the very least, a task requires one thread. You're confusing threads and processes; they're not the same thing. There's one process (Emacs) and zero threads.

That isn't true at all, at least on Linux. All processes have at least one thread, hence each process has atleast 1 associated tid.

The problem is you don't understand what a "thread" actually is. You think it only exists via some t = new thread() call in java or pthread_create or something. Here: >By default, all components of the same application run in the same process and thread (called the "main" thread). If an application component starts and there already exists a process for that application (because another component from the application exists), then the component is started within that process and uses the same thread of execution. https://developer.android.com/guide/components/processes-and-threads So, as I said, a task needs at least 1 thread.

You could wrap this in a package that does this the way vscode does it. All you need is to specify a host, copy over an emacs binary (maybe static? If possible) and a config (the current config? A dedicated one?) and open a new frame with emacsclient that connects to the remote server. Btw there's no need for X forwarding because one could simply forward the emacs socket over ssh. All of this can be automated with some lisp. The only downside I see here is that the new emacsclient instance would be completely separated from your local one.

I'd think getting emacs server running remotely in a way that emacsclient just works is the right approach. Although I do want to be able to be able to edit local and remote files in the same instance of Emacs. I want to do things like insert remote files as links in local org notes, for example. Having a single emacsclient able to seamlessly connect to multiple emacs servers, both local and remote, would be really neat. Heck it might even solve the multithreading problem, since you could fork off an entire emacs server instance for whatever it is you're doing that might mess up everything else. Would be nice for EXWM users to be able to have one emacs server handling the X stuff and then others for file editing and whatnot.

Just re-upping this so it doesn't slip. This is good education for you. A process has at least one thread (the main thread).

Regarding convincing work, a standalone CLI written in python would be easy to install on remote with pip/conda. I’d still vote for a remote server b/c otherwise it is exactly the same as TRAMP and you wouldn’t get any additional benefits. Having a server physically on remote also helps in fast local LSP integration.

> otherwise it is exactly the same as TRAMP and you wouldn’t get any additional benefits Not really, you'd get asynchronicity, which is something TRAMP largely doesn't have.

I personally find VSCode's approach to be terrible. It's so memory inefficient, especially with a lot of users sharing a server.

Let’s assume I’ve got the Emacs binary and my config on the remote machine. Could you elaborate on how you might do this: > forward the emacs socket over ssh In my setup I kept hitting a wall trying to get X forwarding working, so if there’s an alternative I’d like to try that too

I basically agree with this but with the proviso that often one can't forward the emacs socket over ssh for security reasons. ​ Though, honestly, I think just using the server over ssh without x forwarding is good enough. I prefer to discourage mouse oriented features. I think the only thing is that you need to spend sometime getting copy-paste between the various systems at play here to work well and then this should be good enough for most setups.

https://www.reddit.com/r/emacs/comments/ogejpj/comment/h4im2o0/?utm\_source=share&utm\_medium=web2x&context=3

Actually I was wrong, sorry for the confusion. I always assumed that since you can create a server socket, that you could forward it remotely (for example via ssh) and simply have emacsclient connect to it. While this works, emacsclient is not the one that actually creates windows or ttys, it's just going to ask the server to do it (so window creation would happen in the remote server). So it actually works in the opposite way that i initially assumed.

Right now, emacs client doesn't connect to a remote server.

In some ways tramp already has a "remote server": one of various shells, and a large set of one-off perl scripts it sends over the line. Check M-x find-library tramp-sh sometime. But it also has the very complex problem of trying to work in *all possible combinations* of remote machines, shells, versions of shells, perl versions (or not), shell configs, etc., etc. This is a *hard problem*, and so tramp is necessarily quite defensive in its communications. It has to be. If tramp had allowed itself to insist on some "constraints" on the server, it could have saved a lot of trouble. For example: "you have to have Python 3.8 or later with the following packages; pip install tramp-py to get setup". Then tramp could interact with a single well-known and controllable server process, and not spend so much time sending probes and tests and checks to see if the "server" on the other side is going to cooperate or just barf on you. This could all still happen over SSH, but I suspect such a setup would be way more performant, not to mention easier to debug. TRAMP's job has been to write a client for a quasi-random and unpredictable server. Not easy.

Funny, I just spent the weekend trying to get Tramp to work for the nth time and I could not agree more with OP. Tramp is a dead end. Its hideously complex, performs terribly and its super buggy. I respect the miracle that it is to be able to do remote editing without server side cooperation but the limitations are obvious and VS Code shows that strategy is played out. Where I work, a super high tech shop, where everybody logs into their dev boxes from a personal laptop, VS's code remote editing is the killer feature that keeps pulling even the most talented devs, the kind of people who build and program their own keyboards, to VSCode. The few Emacs holds outs are reduced to using SSH and terminal based emacs and dealing with the resulting limitations and latency. I am seriously considered writing my own tramp alternative and I would love to hear from anyone else interested in discussing this.

No, that will perform horribly if your ping is >100ms. You need the editor to live where you are, so it can respond instantly to edit operations, with a server side component running the tools that can only run on the server side (compiling, running a language server, etc)

Still upping this until you admit you are wrong or at least explain why you're not.

As great as it would be, I've given up on VSCode style remote editing in emacs in the short term. Hopefully someday. In the meantime, I wrote: https://github.com/nolanl/rp to continuously sync my local edits to a remote copy of the same source tree. It also allows easily running shell commands on the remote server, making sure the remote copy is fully synced up before the command runs. I've been meaning to write an emacs package to make things like "M-x compile" use it transparently, but haven't gotten around to it yet. For now running builds and tests in a shell have been good enough. Might be useful for others too.