feat: add Lua GC flags #5194
feat: add Lua GC flags #5194
Conversation
| @@ -1109,6 +1203,16 @@ Interpreter* InterpreterManager::Get() { | |||
| } | |||
|
|
|||
| void InterpreterManager::Return(Interpreter* ir) { | |||
| static const uint64_t max_memory_usage = absl::GetFlag(FLAGS_lua_mem_usage_force_gc); | |||
There was a problem hiding this comment.
i would imagine we would like to change these flags without restarting the process.
please add support to "CONFIG SET" as well.
| static const uint64_t max_memory_usage = absl::GetFlag(FLAGS_lua_mem_usage_force_gc); | ||
| using namespace chrono; | ||
| ++tl_stats().interpreter_return; | ||
| if (tl_stats().used_bytes > max_memory_usage) { |
There was a problem hiding this comment.
Consider the following scenario. We have a per thread limit of 100MB
- Lua1 allocates 1MB, does not trigger gc.
- .... Lua99 does not trigger GC and we have like 99MB returned.
- Now we have bunch of scripts created and returned - they allocate small blobs but at some point they cross 100MB so they being GC'ed and the lua memory usage goes around 100MB because the already returned scripts are not GCed.
To summarize - we have a shared per-thread threshold, but we perform GC on lua we currently return that can have only 10KB of allocated memory and most of the allocations are kept in the dormant interperters.
There was a problem hiding this comment.
It's not that I have a proposal, but lets introduce a variable that tracks gc collected memory by counting LUA_GCCOUNT before and after RunGC. In fact, RunGC could return the amount of memory it released.
This way we will be able to estimate GC performance rate.
| @@ -821,6 +821,8 @@ void Service::Init(util::AcceptServer* acceptor, std::vector<facade::Listener*> | |||
|
|
|||
| config_registry.RegisterMutable("pipeline_squash"); | |||
|
|
|||
| config_registry.RegisterMutable("lua_mem_usage_force_gc"); | |||
There was a problem hiding this comment.
that's won't cut it and you have not tested that it works.
you used static const uint64_t max_memory_usage = ... so it will be initialised only once
I've added 2 flags:
I have done some tests with lua_mem_usage_force_gc, and it shows quite good results to my mind, and it works fast.
Regarding the luagc flag, I've tried to use different parameters, according to https://www.lua.org/manual/5.4/manual.html#2.5.1 , and the results were insignificant, but with aggressive parameters performance can be dropped significantly