clojure-lsp integration

[ericdallo]

Hey, with the latest Calva release, I managed to install Calva on my NixOS and test the integration with clojure-lsp 🎉
It works really well, congratulations on the effort on this :)

I created this to discuss how we can improve the integration of clojure-lsp features with Calva, let me know if you prefer a separate issue for that, is just that the things discussed here are not necessarily an issue but something we can discuss if makes sense for everyone/users.

[ericdallo]

Format document using cljfmt via clojure-lsp

Currently the `document-formatting? setting is hard-coded as false in Calva, probably because Calva uses its custom formatting or uses cljfmt directly?
As Calva uses LSP and the protocol itself supports the formatting feature, should we standardize this on Calva and leave it to clojure-lsp the formatting? Is there any formatting setting Calva has today that cljfmt via clojure-lsp doesn't support?
Should we at least provide a setting for users that want to use the formatting via clojure-lsp?

[bpringe]

Calva uses its own fork of cljfmt, partly due to I think one added feature (maybe it was others in the past but cljfmt has added those itself by now... @PEZ knows more). There's a related issue here: #1179. I'm not sure now whether Peter was meaning we could migrate to just using upstream cljfmt or if there are reasons to keep using a fork.

[PEZ]

The reason for the fork is that Calva supports aligning of maps and bindings. This is being added to cljfmt as we speak so soon we shouldn't need this. I'm thinking that in waiting for that to be done, maybe we should implement a workaround in Calva so that we can use the upstream cljfmt directly instead.

That would remove that feature as a blocker for letting clojure-lsp handle formatting. But I still hesitate, for two reasons:

1. We would lose the control of the Ux for formatting. Calva has a bit different Ux for this than VS Code has.

EDIT: For the second reason I moved this out to the root level of the discussion, since @bpringe had already started the broader aspect of this there.

[ericdallo]

Option to override keep-require-at-start?" setting

Maybe we should allow user to override this setting as if anyone wants to keep the first require at the same line as :require and not on the next line.

[bpringe]

What do you mean by option to override? Currently the user can set this in their lsp config, right?

[ericdallo]

Hum, you are right, I forgot that user still can change those things via lsp .config.edn.
Looks good to me :)

[ericdallo]

Support semantic-tokens

This is a recent feature introduced on latest LSP 3.16, clojure-lsp already supports and it's used by lsp-mode(Emacs) already.

Basically, clojure-lsp returns what is the meaning of each token for a document like keywords, functions, variables, comments etc, and the client adds the colors they want for those tokens following the current user theme.

cider does something similar during jack-in on emacs, but with that, we don't need to rely on REPL for this feature :)

[bpringe]

I'll defer to @PEZ here.

[PEZ]

Sounds very interesting. As for CIDER, it also highlights macro calls, iirc. Would this do that? I've often wanted this.

Generally, this sounds like a good idea. I don't think it would conflict with highlighting that Calva does, mainly just enhance it. But let's be a bit specific:

1. Comments, you say. Calva highlights (comment ... forms, would clojure-lsp do that with this feature implemented?
2. Also on comments, how about ignores?
3. What about unbalanced parens?

I'm thinking it could get to be a weird competition, as I wouldn't want to rely on clojure-lsp for some more important highlighting.

[ericdallo]

it highlights based on clj-kondo analysis, so macros, functions keywords, variables would be highlighted. Actually clojure-lsp doesn't handle comments (;;) or block comments ATM so this would not be an issue I believe.

What about unbalanced parens?

It depends if clj-kondo would be able to analyze the code with unbalanced parens, if not, clojure-lsp would not return semantic tokens for that document and the client should use the default highlight. Think about this feature as an enhancement to current highlighting, it should add colors to tokens the editor can't know for sure what is like macros and function/var calls mainly

[PEZ]

I want it!

[bpringe]

There's an overarching question here about relying on clojure-lsp for more functionality than we already do. @PEZ, do you think it's fine to rely more on clojure-lsp for things like formatting and coloring? I definitely see a benefit in doing that, rather than continuing to maintain that related code ourselves, but maybe you see otherwise.

[ericdallo]

Yeah, that makes sense to me, I just thought that could be users that would like to have the formatting via clojure-lsp instead of calva custom, so a setting to use clojure-lsp formatting instead could be a good option for both sides.

[PEZ]

Calva has a command where you can format the current enclosing form. Would that work if clojure-lsp is doing the formatting? I think the option makes sense, just wondering what implications it would have. If this command can't be supported then the Calva would work quite differently depending on this setting. Besides that it might cost to maintain the setting if it is not just a fix the middleware.

Can we make it so that the users wouldn't care if it is Calva natively or clojure-lsp that does the formatting? If we read from the same cljfmt config for instance. In Calva there is a setting for where to pick up the file. We could make that default to where cljfmt is looking for the file. Calva doesn't currently pick up these settings in Leiningen project files, does clojure-lsp do that?

[PEZ]

Example of when relying on clojure-lsp creates a lesser user experience:

I am currently testing a PR that adds a command for switching between test and implementation namespace files, creating the files if they do not exist.

I am testing it both on Windows and on Mac. On the Windows machine (a really cheap, slow, thing with only 4GB of RAM), I am testing it on a super minimal clj-new project. I have never seen the initalization of clojure-lsp complete. On my Mac (a pretty decent Macbook Pro 2019 with 6 cores and 32GB RAM) I am testing it with a fresh project using the Luminus template, where clojure-lsp initialization warms up my computer and takes 2-3 minutes to complete.

We recently removed Calva's own code for adding ns forms to newly created files, and are relying on clojure-lsp for this. Which results in that on my Windows machine I don't get these ns forms at all. And on Mac I need to understand that it won't start creating these ns forms for me until the clojure-lsp init is done. (Which I don't think many users will understand given how seamless the clojure-lsp integration is.)

Is there a way that clojure-lsp could make services that don't need full analysis available before the init is done? That would make it much easier to chose clojure-lsp over native Calva code. Then when init is done enhancements would pour in.

[ericdallo]

That slowness doesn't seems right, you should eck clojure-lsp logs for anything unusual, this should take more time the first time only.

Most of features need to the project to be analyzed that's why we wait to return a initialized response, but formatting is something that could be done at any time I think.
Maybe this could work but it'd need some tests and refactor on clojure-lsp side, WDYT @snoe?

[bpringe]

@PEZ The fact that init doesn't complete on your Windows machine is the most concerning thing to me. It should always be completing for everyone. We should troubleshoot that in an issue (paste the logs as a start).

Ideally (and I thought this was the case or very close to it at least) clojure-lsp should be as reliable/likely to start up as Calva.

[PEZ]

Thanks for starting this discussion!

[kurbatov]

Using default VSCode terminal for clojure-lsp classpath lookup

Today I have tried to open up a clojure project of mine in VSCode after a long break and, after updates, found out that clojure-lsp cannot figure out classpath with the following error:

Classpath lookup failed when running `pwsh -NoProfile -Command lein with-profile +test,+dev classpath`. Some features may not work properly.
Error: Cannot run program "pwsh" (in directory "D:\projects\norm"): CreateProcess error=2, File not found

It turned out that PowerShell is known as the powershell command on my system instead of pwsh that clojure-lsp tries to use.

Thanks to @crimsonhawk47 and #1807, I managed to redefine the :classpath-cmd for clojure-lsp using a .lsp/config.edn file in the project's directory.

I followed the linked issue in the clojure-lsp repository and the corresponding PR with a proposed solution. Even though it solves the issue for clojure-lsp as a standalone project, I think Calva should rather specify the terminal that is configured in VSCode to the clojure-lsp. For example, I would like clojure-lsp to use Git Bash which is set as a default terminal in my instance of VSCode:

"terminal.integrated.defaultProfile.windows": "Git Bash"

So I would like to discuss is there a way for Calva to figure out the command for default VSCode terminal and specify it for the clojure-lsp to use during classpath lookup?

[bpringe]

Oh, I understand better now what you're suggesting. What would the benefit be of making clojure-lsp use the same terminal executable that VS Code currently uses to perform the classpath lookup? Once the change is made in clojure-lsp to use powershell when it's available, the issue you had should be fixed.

I don't see a benefit to adding code to Calva to modify the lsp config (or use some other method) to make clojure-lsp use a different executable, especially since it can be set by the user in the lsp config. Maybe I am missing something, though.

[kurbatov]

What would the benefit be of making clojure-lsp use the same terminal executable that VS Code currently uses to perform the classpath lookup?

It is for consistency reasons. Different shells could be configured differently. For example, I use sdkman in order to manage versions of my build tools and I manage Leiningen with sdkman as well. Sdkman is set up for gitbash and powershell does not recognize lein as a legit command. I have configured VSCode to use gitbash and instinctively expected that the default shell for something launched by VSCode would use gitbash. The update of clojure-lsp that uses powershell will not fix it for me because next time I will see the same message not because a wrong powershell command is used but rather because powershell does not have lein visible in my case.

Yes, I can work around that using the lsp config but given the error message that VSCode provides, it is hard to find the solution.

[bpringe]

I see. Making Calva be able to tell clojure-lsp which shell executable to use would require changes in both Calva and clojure-lsp. Since it seems your case is rare (given that you're the only one that's mentioned this), I'm not sure it's a good idea to make those changes (adding one more thing to maintain in each project) just for this case. However, do you think some documentation could be improved regarding this so that if someone else has this issue they're more easily pointed to the lsp config?

[kurbatov]

I think the whole case deserves a mention in https://calva.io/quirks/ or in the troubleshooting section of this page https://calva.io/clojure-lsp/

[bpringe]

Thanks. Issue created: #1838.