Is there a way to have Eglot pickup the virtual env?

Hi everyone, When I use a virtualenv, it seems that Eglot does not pick it up, which means that I don't get completion and type warning for the packages I installed. Is there an easy fix for that?

Take a look at `buffer-env`

> buffer-env For Python a better (and fully automatic) solution is `emacs-pet` (https://github.com/wyuenho/emacs-pet). If your setup is somewhat "typical" then `pet` will pick it up without having to configure anything and you are good to go. I haven't tried it with `eglot` but it works fine with common Python LSP servers (jedi & pyright) and `lsp-mode` so I don't see why it wouldn't work with `eglot`.

It does. I think your problem is your virtual environment is not on your `exec-path`. How are you creating your virtual environment?

No it doesn't, I use poetry.el which activates the env on its own, even when calling eglot after the env is set, it still doesn't work. I also use `exec-path-from-shell`. It worked fine with LSP but not with Eglot.

I use poetry too but not `poetry.el`. I have to use `exec-path-from-shell` as well on my Debian machine but it only copies environment variables you tell it too. I instead use `direnv` via `purcell/envrc.el`. Direnv has built in support for `venv` and I've submitted a PR to have poetry support built-in as well but you use it with poetry without it being merged via your `direnvrc`. Poetry also allows you to include the virtual environment within your project directory rather than the default location. I've experimented with both locations and both are found.

Mind to share your config?

Sure, which part are you interested in? It is quite a simple config.

Do you launch emacs from within the project directory, as opposed to starting it from a launcher or something? That way the emacs process will inherit all the env vars that are set up by virtual env.

Specifically how are you defining "the virtual env"? And how can something (such as Emacs) read it?

That's interesting, how did you check that the locations were found?

I have `direnv` export a few ENV variables namely: $VIRTUAL_ENV which gives the full path to the virtual environment, $VENV_BASE which gives the basename of the virtual environment path (I use this to update my $PS1 in vterm (or a standard terminal for that matter) to indicate when a virtual environment is active/activated), and POETRY_ACTIVE to indicate poetry is active i.e. creating my virtual environment, managing my project dependencies etc. I use envrc.el to integrate direnv with Emacs. It tells me if direnv was successful on opening a file in a direnv managed directory. I can pop open vterm within my project directory and confirm the virtual environment is on my path and that the python within the virtual environment is the active python.

I tried it out as well but it doesn't seem to work either. My virtual environment is picked up just fine and the code runs fine as well but Pyright can't seem to find the stubs, the completion doesn't work properly.

I've only used pylsp but I can try with pyright tomorrow.

Thanks for the help!

One thing I do remember off the top of my head which is my mind a bug in poetry. Only virtual environments created with the python used to install poetry work. I use `asdf` to install different versions of python. This is something independent of Emacs. An issue was raised on the poetry repo and a fix implemented but I'm not sure if it was released yet. Edit: [some context](https://github.com/orgs/python-poetry/discussions/6978)

I made a mode to handle execution configurations like in pycharm exactly because I was having similar issues and none of the python packages seemed to offer the same thing as pycharm configurations. The closest one was dap mode, but I do not want to run a debugger every time I want to run my program. Have a look if it might help you out, you can specify your virtual environment path and it will load it before execution and then deactivate it (this is handy if you are working on multiple projects and your virtual envs are not all in the same place): https://github.com/andcarnivorous/pyconf

I quite interested in how poetry direnv and exec-from-shell works together, also, your python lsp server is installed globally or per project? Do you use a particular .dir-env.locals?

Eglot doesn't provide completions on its own. Maybe you mean your language server does not pick up your venv. Which LS are you using? You can configure \`eglot-workspace-configuration\` as a function or via .dir-locals to point your LS at the correct venv/python executable (with exact config depending on the server). Edit: I see below you use pyright. You need to return something like: (list :python `(:venvPath ,venv-path :venv ,venv-name :pythonPath ,python-interp))

I have [tool.pyright] include = ["*"] exclude = ["**/node_modules", "**/__pycache__", "src/experimental", "src/typestubs" ] defineConstant = { DEBUG = true } venvPath = "/home/ME/Code/Python/misc/" venv = ".venv" reportMissingImports = true reportMissingTypeStubs = false pythonVersion = "3.10" pythonPlatform = "Linux" In my `pyproject.toml`, which Pyright should pick up if there is no `pyrightconfig.json`, as per the docs. I don't really understand what's going on since it was behaving fine with `lsp-mode`. Edit: So, pyright proper **does** pick up this without issues. But `eglot-workspace-configuration` stays `nil` for some reason, which seems to be the issue.

I'm using: 1. [`asdf`](https://github.com/asdf-vm/asdf) a CLI tool that can manage multiple language runtime versions on a per-project basis, extendable via plugins + a. [`asdf-python`](https://github.com/asdf-community/asdf-poetry) an asdf plugin to manage python versions on a per-project basis (uses [`pyenv`](https://github.com/pyenv/pyenv) under the hood) + b. [`asdf-direnv`](https://github.com/asdf-community/asdf-direnv) an asdf plugin to integrate `asdf` and `direnv` + 2. [`purcell/exec-path-from-shell`](https://github.com/purcell/exec-path-from-shell) to make some ENV variables visible to Emacs + 3. [`direnv`](https://github.com/direnv/direnv) to load/unload ENV vars depending on the current directory (avoids the need to manually activate a virtual environment) + a. [`purcell/envrc`](https://github.com/purcell/envrc) to integrate `direnv` with Emacs, alternatives include [`wbolster/emacs-direnv`](https://github.com/wbolster/emacs-direnv) and [`astoff/buffer-env`](https://github.com/astoff/buffer-env) 4. [`poetry`](https://github.com/python-poetry/poetry) to create virtual environments and manage project dependencies (Note, `asdf-direnv` can also be used to install `direnv` but I don't use it for that. I instead just use my system version. There is also an `asdf-poetry` plugin to install different versions of `poetry` but again I had no need for that.) I use `exec-path-from-shell` to copy the `asdf` "ASDF_CONFIG_FILE" "ASDF_DATA_DIR" "ASDF_DIR" ENV variables to `exec-path`. My relevant Emacs config is as follows: (use-package exec-path-from-shell :when (or (memq window-system '(mac ns x)) (unless (memq system-type '(ms-dos windows-nt)) (daemonp))) :init (setq exec-path-from-shell-arguments '("-l")) (setq exec-path-from-shell-debug t) :demand t :config (dolist (var '("ASDF_CONFIG_FILE" "ASDF_DATA_DIR" "ASDF_DIR" "GNUPGHOME" "GPG_TTY" "LEDGER_FILE" "SSH_AUTH_SOCK")) (add-to-list 'exec-path-from-shell-variables var)) (exec-path-from-shell-initialize) ;; Need to force the setting of the GPG_TTY environment variable on my machine. (setenv "GPG_TTY" "dev/pts/0")) (use-package envrc ; Buffer-local direnv integration. :if (executable-find "direnv") ; Only load envrc if direnv is installed :demand t ; because I lazy load my packages by default :config (envrc-global-mode)) (use-package eglot ; Emacs polyGLOT - a LSP Client. :bind (nil ; be explicit that there is no global bindings, corrects an indentation issue in use-package :map eglot-mode-map ("C-c f" . eglot-format) ("C-c h" . eldoc) ("C-c o" . eglot-code-action-organize-imports) ("C-c r" . eglot-rename)) :config (defun my-eglot-format-buffer-on-save () (if (and (project-current) (eglot-managed-p)) (add-hook 'before-save-hook #'eglot-format-buffer nil 'local) (remove-hook 'before-save-hook #'eglot-format-buffer 'local))) (add-hook 'eglot-managed-mode-hook #'my-eglot-format-buffer-on-save)) (use-feature python ; Python editing. :config ;; Default to IPython. Otherwise, prefer the versioned Python 3 binary to ;; handle systems that have the unversioned one point at Python 2! (cond ((executable-find "ipython3") (progn (setq python-shell-buffer-name "IPython") (setq python-shell-interpreter "ipython3") (setq python-shell-interpreter-args "-i --simple-prompt"))) ((executable-find "ipython") (progn (setq python-shell-buffer-name "IPython") (setq python-shell-interpreter "ipython") (setq python-shell-interpreter-args "-i --simple-prompt"))) ((executable-find "python3") (progn (setq python-shell-interpreter "python3") (setq python-shell-interpreter-args "-q"))) ((executable-find "python2") (progn (setq python-shell-interpreter "python2") (setq python-shell-interpreter-args "-q"))) (t (progn (setq python-shell-interpreter "python") (setq python-shell-interpreter-args "-q"))))) Note, I don't use `eglot-ensure` to avoid unnecessary errors opening a python file and the lanaguage server is unavailable. `M-x eglot` is sufficient when I want to engage the language server. This is an approach echoed by the Eglot author. See the [direnv wiki](https://github.com/direnv/direnv/wiki/Python#poetry) where I wrote what I add to my `direnvrc`. This has also been submitted as a PR for consideration to have `poetry` support built-in. I describe my two workflows there too but they don't mention my use of `asdf` (as it wasn't relevant to `direnv` users) but all that differs is that the `echo` command becomes `echo -e 'use asdf\nlayout poetry' > .envrc` in each case. See the example on the [direnv wiki](https://github.com/direnv/direnv/wiki/PS1) where I show how I update my `$PS1` so it is visible in `vterm` for example or a terminal outside of Emacs. Also, note `poetry` does not set the path or activate the virtual environment if the active python version differs from that used to install `poetry` - see [here](https://github.com/orgs/python-poetry/discussions/6978). This has been fixed upsteam but not yet released. I use `pylsp` installed per virtual environment. I put together this [`.dir-locals.el`](https://gist.github.com/doolio/8c1768ebf33c483e6d26e5205896217f) for use with Eglot.

That’s not an issue. You don’t *need* to use eglot to configure the server, but you *can*. If you read the beginning of the eglot events buffer you can see what the server is doing for setup (under eglot's control). E.g. I see Setting pythonPath for service ....

OK, so I created a demo-project and installed `pyright` into the `poetry` created virtual environment. Output from `vterm` below: $ poetry env info --path /home/doolio/.cache/pypoetry/virtualenvs/demo-project-k8-nqTjx-py3.10 $ which python python is aliased to `python -q' python is /home/doolio/.cache/pypoetry/virtualenvs/demo-project-k8-nqTjx-py3.10/bin/python python is /home/doolio/.local/share/asdf/installs/python/3.10.7/bin/python python is /home/doolio/.local/share/asdf/shims/python $ python --version Python 3.10.7 $ which pyright pyright is /home/doolio/.cache/pypoetry/virtualenvs/demo-project-k8-nqTjx-py3.10/bin/pyright With the following simple snippet: def greeting(name: str) -> str: return 0 I get the following reported by `pyright`: foo.py 1 note Pyright: "name" is not accessed foo.py 2 error Pyright [reportGeneralTypeIssues]: Expression of type "Literal[0]" cannot be assigned to return type "str"   "Literal[0]" is incompatible with "str" Is that sufficient? Edit: my `exec-path` when visiting a demo-project file is as follows: ("/home/doolio/.cache/pypoetry/virtualenvs/demo-project-k8-nqTjx-py3.10/bin/" "/home/doolio/.local/share/asdf/installs/python/3.10.7/bin/" "/home/doolio/.local/share/asdf/plugins/python/shims/" "/home/doolio/.local/share/asdf/shims/" "/home/doolio/.local/share/asdf/bin/" "/home/doolio/.local/bin/" "/usr/local/bin/" "/usr/bin/" "/bin/" "/usr/local/games/" "/usr/games/")

Thanks for the detailed answer, I get something similar! The last task right now is to get libraries stubs for proper type hinting support.

I'm just a beginner programmer (started with python) . Do you have a snippet I could use to test if that functionality is available with my setup?

Sure, you could try to download a library like `requests` or `pandas` in your setup. Just do `poetry add pandas` in the directory where your `pyproject.toml` is for example. Now, open a python file with, for example. import pandas as pd df = pd.DataFrame({"A":[1, 2, 3], "B":[4, 5, 6]}) Pyright should complain that there are missing stubs/types for pandas.

\> Pyright should complain that there are missing stubs/types for pandas. In what way does it complain? If I place point pd I get: (module) pd But if I place it over DataFrame I get: DataFrame: unknown Is that what you mean? Trying to understand this further it appears the issue is with pandas. If I do a similar test with numpy (also installed when I installed pandas) or requests I don't have that issue. I get completion suggestions, documentation on hover etc. Can you confirm? I think the issue is that pandas provides its stubs via a separate package namely [pandas-stubs](https://github.com/pandas-dev/pandas-stubs). It seems some packages provide their stubs in a separate package i.a.w [PEP561](https://peps.python.org/pep-0561/#stub-only-packages). If I install this package I get completions etc. as you might expect. (Note, in my pyproject.toml I had \`python = "\^3.10"\` and I had to adjust this to \`python = ">=3.8,<3.12"\` to be able to install pandas-stubs.) In saying that after installing pandas-stubs when I have point on pd I still only get: (module) pd in contrast to np for numpy where truncated documentation on what the module provides. This could well be due to [this section](https://github.com/pandas-dev/pandas-stubs#differences-between-type-declarations-in-pandas-and-pandas-stubs) on the pandas-stub github repo.

Yep it is. It isn't an issue with `pyright` itself, it means that the stubs need to be installed for a specific package. So packages like Numpy have to good idea to include their own stubs within the package, but it's pretty rare otherwise. It's a bit annoying because I get non-sensical type errors with the LSP but it is what it is.

Ok, but one would have the same issue with `lsp-mode`, right?

For some reason, I had fewer issues. I don't know why.