noob question about tree-sitter in the presence of lsp-mode

I saw the video from the tree-sitter creator and see how it would be a great improvement over regex. However, given that things like lsp-mode exist, what does tree-sitter bring to emacs? Won't clangd have the most accurate information about your c++ file than whatever tree-sitter is doing?

Well yes, but lsp-mode isn't used for syntax highlighting which is the biggest strength of tree sitter from what I've heard I'm curious generally as to how people use tree sitter : i've never felt the need yo use it since the usual major modes + lsp seem to cover my needs

I set (lsp-semantic-enable-tokens t), which I thought allows lsp-mode to take over syntax highlighting, but now that I compare the syntax highlighting to a buffer without lsp-mode, I see that you are right. It seems like the semantic-enable-tokens goes further with allowing one to choose colors for various aspects of syntax highlighting, maybe it is a wip?

I don't think there is anything like [code folding](https://github.com/emacs-tree-sitter/ts-fold) with eglot/lsp-mode, is there?

Sure there can be folding hints provided by the LSP, I think neither eglot nor LSP support it https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#foldingRangeClientCapabilities

Treesitter font lock is just one thing. Folds, syntactic text objects are another and further boost productivity and usability

Do you know why they don't support it?

lsp-mode supports it via lsp-origami.

There was a discussion in vscode community that having full-featured syntax highlighting over lsp won't be fast enough, thus only the semantic one.

>That is exactly what I was trying to ask. Why can we not get that information from say clangd in the case of c++? clangd provides this information already.

Thanks. I did not know that.

I don't know how accurate I am to the absolute correct answer to this question but still I am making this point for the clarity. Tree sitter is a new way to parse plain text files incrementally and by understanding the genral grammer of the text file. A language server on the other hand provides IDE like capabilities such as code completion, snippets, auto checking of source code etc to basic text editors. A language server has nothing to do with how the editor parses the source code. Sure it has lots of info about the source code, but not the grammer specifics that a editor can find more useful. A language server generally operates inside a project directory with project dependencies, lockfiles, VCSs, etc. Also you can't have a language server for things like config files, markdown documents, org files, toml, json etc. Whereas treesiter only operates on the file that it has been attached with. It doesn't need a project directory or anything like that. And yeah! you have treesitter support for markdown, config files, toml, json, yaml and the list goes on.

One thing, treesitter is a program dedicated to parsing languages that uses language grammar files to parse syntax, so the ease of writing a new treesitter grammar is dramatically easier than implementing the LSP. Second, most LSP implementations are done in the same language, so your LSP implementation is often only as good as your compiler, and has the overhead of being a client/ server, so you of necessity have message passing back and forth between the server and your editor. I do not believe that treesitter has that same kind of protocol overhead. Also some language servers just really suck and are super slow and if I had to rely on that for font-locking, I’d quit.

You can think of LSP as something that works on the "large". Tree sitter works on the "small". LSP is like a smart index of your entire project to power goto-definition and auto completion. Tree-sitter just makes a tree of objects representing the text in your buffer. There can be a some overlap in the features they power but that's coincidental. Tree-sitter by it's nature is better suited to power syntax highlighting. And better suited to power structural edits (barf, slurp, transmogrify). As far as overlap, one case would be highlighting all instances of the variable your pointers is on. This might be possible with LSP but it would be super overkill to send a JSON message to an external server just to highlight a few local references on the screen. Tree sitter would be best for this. But if searching the entire project for variable references LSP would make sense as that's the "large".

I honestly can only speculate that the implementation is not trivial (be it in eglot or lsp-mode) and there are only that many ressources working on it. As a side-node and question. What Emacs inbuilt methodology should "folding" use? The closest I can think of would be hide-show mode. Did you try that? My unfortunate experience is that it works ok on rather small files. The default shortcuts - well, you can re-assign them. hs-mode performance is rather bad for large files though. I speculate and hope that improvements in Emacs 29 (speed-up of overlays away from O(1) would make hs-mode more "usable".

re syntactic text objects: https://github.com/mickeynp/combobulate

That's great! However I will wait using it until the author works on his disclaimers > NOTE: Because combobulate is alpha software it has hard dependencies on avy, multiple-cursors and hydra.

I don't know that Mickey Petersen will do much more with confabulate. My point in referencing it was to show how a well-known emacs documenter and popularized approaches and thinks about tree-sitter. Tree-sitter in emacs is in its infancy. I expect we will see a lot of great tree-sitter enabled functionality emerge from the emacs developers over the next release.