How can virtual formatting be achieved in Emacs?

I'm interested in virtual formatting of code, that is that the code doesn't contain (almost) any formatting, and the editor parses the code and lays it out according to its structure and some layout rules. [You can read more about virtual formatting on the site of XMLQuire.](https://qutoric.com/xmlquire/virtualformatting.html) Virtual formatting isn't that the editor parses the code and formats it by inserting whitespace; the text is not modified, it's just laid out on the screen to reflect its structure. How could that be achieved in Emacs? I didn't find any editor capable of that for multiple languages, and Emacs has surprised me multiple times with what is possible in it, so I have hope that it is possible in Emacs.

Text properties and/or overlays (refer to the elisp manual) can be use to change the appearance of text in a buffer; so yes, this can be done in Emacs in principle. I'm not aware of any implementations of what you're asking for (but also haven't ever looked). If you plan on implementing this yourself, you'll likely also need `after-change-functions` to re-parse modifications to the buffer, and update the formatting accordingly.

I bet tree-sitter makes this easier than ever. ​ There are modes that provide an alternate view of the underlying data, significantly so in some cases. 'hexl-mode' provides a hex-editor interface to the underlying binary data, perhaps there is some inspiration in the code of the mode?

I do some of that. For instance: \-It doesn't matter how end of line comments are indented - I force their display column. \- I don't care what indent step code uses - I set a scaling factor for beginning of line indentation that looks nice on each display. \- Code displays the same for me whether it uses spaces around operators or not. I put virtual visual whitespace around them (in the range of half a space) according to precedence.

That's interesting. Where can I learn more about that?

It's all done via font lock mode. For instance, for controlling indentation, I define a font called font-lock-indentation-face, and control its size using normal emacs customization. I then add this to my font-lock keywords: ​ ("\^\\\\(\\\\s-+\\\\)" 1 font-lock-indentation-face t )

I don't have a complete answer, but I'm super interested in this topic, and might be able to give a few pointers. I'm really interested in cataloging the various kinds of Emacs UIs that exist and pushing what's possible. Here's my high-level take on Emacs display capabilities: The vast majority of Emacs modes have (what I'll call) their *primary content* in a buffer, and display something pretty similar to that when displayed in a window. The next layer of display sophistication is font-locking, and then one layer beyond that are text overlays and text properties. With that functionality you can control faces, links / buttons, selective display, images, alignment, inline spacing, (non-)interactivity, local keymaps, line height, etc. But you're pretty much stuck with content being displayed in the same order that exists in the buffer (one interesting, slight exception here might be the `before-string` and `after-string` overlay properties). Then you have a whole class of modes where the buffer contents are really just an output format (a view, in an MVC-ish sense). Notable examples include: magit, dired, customize, hexl, tabulated-list, anything compilation-esque (`compilation-mode`, `grep-mode`, etc.). Generally speaking, their state is either primarily in memory, on disk, from a process, or across the network (obviously there's overlap between many of those things), but they're not likely to write their buffer contents directly to file and then read them back. Some interesting things you should definitely look into: * Ewoc (called [Abstract Display](https://www.gnu.org/software/emacs/manual/html_node/elisp/Abstract-Display.html) in the Emacs Lisp Manual. This is, AFAIK, is about as far as anything goes in trying to do a two-way binding between arbitrary data and rendering format. * `forms-mode`. Seems like it has some rough edges, but it takes data from one file, transforms it to a display format via a definition file, and then writes records back to the original format after edits. * [File Format Conversion](https://www.gnu.org/software/emacs/manual/html_node/elisp/Format-Conversion.html). Functionality to convert file contents to buffer text (in arbitrary ways!) and back. I'm not sure what modes use this, but it seems like an interesting tool for a mode where display differs highly from on-disk format. But you'd still have to figure out the interactivity story. Finally, the power tools (all with major caveats): * Child frames. Overlay windows (even borderless) on top of other windows. Seems like it could be used for arbitrary text layout (performance and convenience of editing be damned). * Xwidgets. Theoretically display anything you want (in a box). But today you can "only" display a browser. * SVG images. Draw any vector graphics you want (including text!). In a box (see the theme?). * [Dynamic Modules](https://www.gnu.org/software/emacs/manual/html_node/elisp/Dynamic-Modules.html). Load any damn code you want. IIRC, I've done some arbitrary drawing using macOS functionality at this level, but at this point you're in crazy territory and nothing you do is going to mesh well with anything else in Emacs. I hope that's helpful! I'd love to chat about the above with anyone if I missed anything, or have ideas about this topic.