Support for module-style includes?

[Toqozz]

How are you using the lua-language-server?

Other

Which OS are you using?

Windows

What is the issue affecting?

Diagnostics/Syntax Checking

Expected Behaviour

Lua files from my project are found automatically without adding a workspace library path.

Actual Behaviour

Incorrect diagnostics due to LSP seemingly being unable to find the lua files in the path. Also go to definition and such don't work.

Reproduction steps

.

Additional Notes

I have a lot of files in my project which I import like so:

import 'game.physics'
import 'game.ball.request'
import 'game.world'

Note that we use import instead of require. It doesn't do anything special regarding the paths, and I have the following setting:

"Lua.runtime.special": {
	"import" : "require"
}

These paths are relative to the workspace, plus some folders. The actual paths are like so:

<workspace>/src/script-game/game/physics.lua
<workspace>/src/script-game/game/ball/request.lua
<workspace>/src/script-game/game/world.lua

From reading the documentation, it seems like the default Lua.runtime.path should pick these files up no problem, but I get diagnostics like "Undefined global "physics"", and I can't go to definition or get completions for those files (details omitted, sorry):

---

If I then add a config for Lua.workspace.library, I can make it work:

"Lua.workspace.library": [
	"<workspace>\\src\\script-game"
],

But this feels wrong. These are my files and they're part of my project. I shouldn't need to add them as a library path, should I? Additionally, I could only get it to work as an absolute path, so I need to update this whenever I switch projects. I tried "src/script-game" and "src\script-game" and both reverted back to the broken behaviour.

Am I missing something? Do I need to set something custom in Lua.runtime.path?

Log File

Unfortunately I can't provide this as most of it is under NDA.

[tomlau10]

By default, runtime.path is set to ["?.lua", "?/init.lua"], and if you have also set runtime.pathStrict to true, then LuaLS will only search the 1st level of your workspace directory 🤔

i.e. when you do

import 'game.physics'

=> it will only look for <workspace>/game/physics.lua or <workspace>/game/physics/init.lua

---

I would suggest you look into your log files to see if pathStrict is turned on or not. 😕

TBH I would turn it on personally, because if pathStrict is false, LuaLS might get confused when there are multiple modules ending with the same base name.
And then I would set the runtime.path according to my project's directory structure.

Do I need to set something custom in Lua.runtime.path?

In your case, according to your description, I would suggest setting it to ["src/script-game/?.lua"].
And in case you don't know already, you can have a .luarc.json in your project, so you don't have to mess up your global VSCode user settings 👀
https://luals.github.io/wiki/configuration/#luarcjson-file

[Toqozz]

Checking the log file, pathStrict is definitely false.

I've turned it on and set my path as you describe, but still nothing :(

"Lua.runtime.pathStrict": true,
"Lua.runtime.path": [
	"src/script-game/?.lua"
],
"Lua.runtime.special": {
	"import" : "require"
}

And in case you don't know already, you can have a .luarc.json in your project, so you don't have to mess up your global VSCode user settings 👀

Thanks, I guess I can work around it by doing that for now.

[tomlau10]

I have run out of ideas... 🙈

I just tried to setup a dummy workspace folder structure like you described, and I cannot reproduce this issue 🤔

• the import keyword works as expected
• the game.world class type is defined correctly
• completion for the world module works as well

contents for each file in this test setup

• .luarc.json

{
    "$schema": "https://raw.githubusercontent.com/sumneko/vscode-lua/master/setting/schema.json",
    "hint.enable": true,
    "hint.setType": true,
    "runtime.pathStrict": false,
    "runtime.path": [
        "src/script-game/?.lua"
    ],
    "runtime.special": {
        "import": "require"
    }
}

• src/script-game/game/world.lua

---@class game.world
local world = {}

function world.hello()
    return "hello world"
end

return world

• main.lua

local world = import "game.world"
world.

[Toqozz]

Ok, I've tried your demo, exact same setup.

I think I've found the issue. When I said import doesn't do anything special, I made a mistake. It seems like it actually loads a package and inserts it into the calling environment.

So these 2 snippets are roughly equivalent (I think):

import 'game.world'

local world = require 'game.world'

Sorry, I should have noticed that.

It does seem that even though I get the 'undefined global' diagnostic I can still hit go to function on hello.world() and it'll take me there. This also happens sometimes in my real project, but it doesn't always manage to take me there.

Is there some way I can tell the language server about this?

[tomlau10]

It seems like it actually loads a package and inserts it into the calling environment.

AFAIK, LuaLS doesn't have a load package and insert concept.
It will just search through and preload all files in your workspace and configured library paths, except if the folder is excluded of course.

For example, if the src/script-game/game/world.lua is using global variable:

---@class game.world
world = {}

function world.hello()
    return "hello world"
end

• then even if you don't write import "game.world" in main.lua, you will still have this global variable world

So these 2 snippets are roughly equivalent (I think):

import 'game.world'
local world = require 'game.world'

Ah~ now I see what you mean, but unfortunately they are not the same in the view of LuaLS 🙈 .

• unless the world is defined as global variable in your files, otherwise with or without import 'game.world', LuaLS will still thinks that world is an undefined global.

[tomlau10]

Is there some way I can tell the language server about this?

Need more on your actual library/framework implementation to give better suggestions 😕

• do your library files use globals or not?
• or are they implemented like my demo using a module namespace object?
• do those library namespace have ---@class xxx annotation or not?
• are you willing to modify your codebase (eg change them to use global, or add local {module} = before import ?

---

Without further details, the following is the least intrusive way to add global type annotation to the modules:

• use a separate meta definition file such as global.d.lua
• you define all the "dummy" globals there, and point them to corresponding module using ---@module xxx annotation
  https://luals.github.io/wiki/annotations/#module

here is an example modified from the above demo setup:

• .luarc.json
  same content
• src/script-game/game/world.lua

-- i assume you library have no annotation, and follow the `return module` practice
local world = {}

function world.hello()
    return "hello world"
end

return world

• globa.d.lua

---@meta _ # an "_" means this is not a module and will not show up as suggestion when you do `require "xxx"`

---@module "game.world" # this makes the following `world` global variable becomes `world = require "game.world"`
world = {}

• main.lua

import "game.world"

world.<try trigger completion here>

---

edit: oh wait~

all the above doesn't explain why everything starts to work by just adding "<workspace>\\src\\script-game" in the library setting 😕 😕

[Toqozz]

Thanks for the detailed replies! This might be the most helpful anyone has been on a GitHub issue for me.

I'm not sure on some of the answers here, since I'm mostly just used to our "flavour" of lua. For instance, I've never used ---@module before.

• do your library files use globals or not?

No, not like you show at least. Here's what the core function of import seems to do:

local callingEnv = getfenv(2)
if callingEnv == _G then
    error("attempting to import into the global environment")
end
callingEnv[packageName] = P

• or are they implemented like my demo using a module namespace object?

They are much more basic than that:

-- src/script-game/world.lua
function hello()
    print("hello world")
end

-- src/script-game/main.lua
import 'world'
world.hello()

• do those library namespace have ---@Class xxx annotation or not?

No, see above.

• are you willing to modify your codebase (eg change them to use global, or add local {module} = before import ?

Unfortunately no, that would mean changing thousands of files.

---

all the above doesn't explain why everything starts to work by just adding "\src\script-game" in the library setting

Yeah. Maybe the diagnostic system is just failing to diagnose once it crosses some number of files? I'll try using that option again, and making sure the diagnostic doesn't return to be sure that it actually is fixed by that.

EDIT: Yeah, today adding those files to the workspace doesn't seem to help with the diagnostic errors :(. I swear it was fixed the other time.

[tomlau10]

With your further explanation, now I have a basic understanding of how your framework works 👀

• you library modules define functions in a "global" style
  but actually they are not globals - they are like in lua 5.1 / LuaJIT using the module() style:

module("world", package.seeall)

-- this function will be under the package `world`
-- and should be seen as `world.hello()` instead of just a global `hello()`
function hello()
    print("hello world")
end

But unfortunately LuaLS doesn't support this AFAIK, it doesn't support getenv/setenv.
And it will just treat all defined functions as global functions.

---

EDIT: Yeah, today adding those files to the workspace doesn't seem to help with the diagnostic errors :(. I swear it was fixed the other time.

Judging from the given snippet, I doubt if it ever works at the beginning 🙈 because LuaLS will just treat all your library functions as globals.

I believe LuaLS doesn't support this kind of framework (defining functions in global styles, but they are actually under some namespace)
Or at least I have no experience in how to setup LuaLS in such situation.
Sorry that I am unable to help you further ☹

[Toqozz]

All good, thank you for your help :)

I guess I'll just have to disable those diagnostics and live with it.

Someone can close the ticket if it's not planned to support this.

[CppCXY]

I know some people use the setfenv function/_ENV variable to create functionality similar to module and import. I've even done this myself in the past, but ultimately it doesn't feel as good as simply using require. Moreover, plugins now have excellent support for require—you can either write paths manually or use auto require—so creating functions like import is really unnecessary. Of course, I understand you're unlikely to change your existing code because of this point. The solution mentioned above seems to be the best approach, although modifying the plugin is also possible if you have the capability. Given the current maintenance situation, We probably won't provide support for this.