-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building a shared global environment #13445
Comments
One rather large issue I see with the approach you've highlighted is that if you're using mix like this, it can only compile things when they've been synced to disk. Lexical doesn't require this now, and gives errors when they happen, as opposed to when the user saves. I think changing this would be a step backwards in developer experience. I really have come to enjoy the as-you type compilation and would like to increase its prevalence rather than eliminate it. Also, using the LSP file watching mechanism is a bit fraught. Emacs, for example ships with the ability to monitor only 1000 files (at least on MacOS), which gets exhausted quite quickly (this is editor-wide, and editors can run multiple language servers). Worse, in order to increase this, you need to recompile Emacs, and then disable some system protections, which requires rebooting your computer four times. I doubt many emacs users will do this, so this solution will exclude Emacs users, at least on MacOS. If you're thinking "hey, we should only watch certain files to reduce the number of watched files", then emacs has your back too (at least with lsp-mode). It ignores that request and watches every file in your directory anyways. I'm an Emacs user, so i'm most familiar with it, I'm not sure how other editors implement or manage file watching. I would hope they do a better job. Currently, Lexical takes an indexing approach to look at your source code --it doesn't make heavy use of compilation tracers, and actually runs mix tasks to compile and get diagnostics. It just places the artifacts in a different directory. This is slightly annoying and slow when you start the LS in a clean directory since we have to get deps, etc. but it's mostly solved, and does use the standard tooling. Elixirls behaves similarly. Another question: If your project and the LS share a compilation directory, then the project will get compiled a lot more than it would be presently. Lexical, for example, compiles the current file on every keypress. Will that interfere with tools like phoenix live reload?
This seems nice, though I think i prefer the approach that the I'm not familiar with |
Hi @scohen, thanks for the input!
I would think those approaches are orthogonal. We can build the global environment, so you can fetch global information about the project (to create stuff like workspace symbols) and find references, but still use Exactly as you said here:
The approach would not change. The idea, however, is that Elixir will provide you with even more information upfront, so you don't have to use any compilation tracer and, hopefully, not even need a separate directory for LS builds. :)
Ah, that's a shame. So we would also need to figure out a mechanism to communicate between processes which files have changed.
Sorry, that's just a random name I came up with. It doesn't mean anything right now. |
Omg, that's hilarious, i spent about 45 minutes looking up what a termdb file is, and could not, for the life of me, figure out how that format would help here. 🤣 They actually exist, and are related to termcaps, and store information to display different escape sequences. I'm still having a little difficulty understanding how the language server and the normal execution environment will coexist. Is the idea that the language server shell out to I'm also concerned about having to support two ways of doing things. These changes are welcome, but a language server needs to support older versions of elixir that don't have these features, so we'll need to keep the internal compilers in addition to integrating the elixir-specific databases. |
Sorry 😱
I am still trying to figure out the details myself. Let's start with the problem and we don't even need to talk about language servers, as the main problems exist within Elixir + Phoenix today. If you run To solve this problem, I would make it so Mix starts a UNIX socket under _build/ that acts as a server. This server will perform both locking and pubsub. If you have more than one instance running Mix, only one of them wins and act as the server. Now, when What about language servers? One of the reasons language servers have their own build directory is to work around the problems above. This means every project needs to be compiled twice, one for the language server and another for the user. For projects with 100+ deps or 1000 files, this can be an issue. If we introduce locking and pubsub, language servers could leverage this too! They will subscribe to pubsub events and now how the project changes. HOWEVER, another reason why language servers have their own _build is because they need to compile code with their own compilation tracers. So if we want to solve this problem, Elixir also needs to build the database (or .termdb files lol) that language servers build during compilation. You would still be able to run Code.with_diagnostics and so on, while you listen to pubsub events to know which modules have been added or removed.
This is definitely a challenge but we will need to figure how to continue improving. Given we have 3 language servers, maybe some of them would be fine with a "latest Elixir" policy. Dunno. |
I didn't know that was on the table, this is an interesting approach, and definitely worth pursuing. |
Hi @lukaszsamson, @mhanberg, @scohen, and @michalmuskala!
We have had several discussions in the past about improving Elixir for language servers. The discussions usually centered around two topics:
The global environment which contains all modules and functions, alongside their imports, aliases, requires, etc (note that not all modules are compiled with tracer events, such as the ones from Elixir or Erlang modules, so there is still need to fallback to runtime reflection APIs)
The buffer environment which is obtained by analyzing the currently open file and only makes sense within the current file (variable definition, module attributes)
In order to build the global environment, language servers need to add their own compilation tracers, and customize compilation. In turn, that requires a custom build directory and the need to compile all code at least twice. I would like to discuss the option of having Mix build this shared global environment instead.
In a nutshell, the idea is that Mix in dev mode will either add a separate .termdb file alongside each .beam file or augment the .beam files with an additional chunk. This way, language servers can rely on the regular code compilation to generate all artifacts they need and then use the file watching mechanism to observe when such files are added or removed. You can then load such files into another database of your preference. Another file will be made available with compilation diagnostics as well, so they can be shown in the UI.
Here are questions for you:
Assuming we are happy with this, we should discuss:
Thank you for your time.
PS: the language server can also just run
mix compile
to trigger a new compilation. However, we need to add locking to make sure we don't have concurrent compilations, @lukaszsamson plans to explore this feature too.The text was updated successfully, but these errors were encountered: