Implement Grafana Query Builder

[archef2000]

Add support for all operations of LogsQL in the Grafana Query Builder.
@Loori-R

#48

[Loori-R]

Tested on Grafana 11.3.1.

An error occurs when adding filters:

1. Open Explorer
2. Go to Operations → Filters → Words
   This issue happens not only with Words, but also with other filters.

[archef2000]

I am currently on 12.0.2 and everything is working will test version 11

[archef2000]

The problem exists up to including v 11.4.6 and is fixed in 11.5.0.
The source problem is that the custom Field Editors get undefined as props

[Loori-R]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana.
The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version.
What do you think, @dmitryk-dk, @hagen1778?

[Loori-R]

@archef2000, Could you please take a look at the tests and fix them?
Here’s the link to the failing workflow:
https://github.com/VictoriaMetrics/victorialogs-datasource/actions/runs/16075336610/job/45477811316?pr=329

[dk]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana.
The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version.
What do you think, @dk, @roman?

In my experience query builders are fairly hard to use manually, - click click click hundred times, - I prefer plain old text for that. But what do I know, I am probably a wrong dk :)

[Loori-R]

In my experience query builders are fairly hard to use manually, - click click click hundred times, - I prefer plain old text for that. But what do I know, I am probably a wrong dk :)

Oops, looks like I tagged the wrong person 😅
Thanks a lot for sharing your thoughts - and sorry for the ping!

[dmitryk-dk]

It looks like the issue is with the Combobox component - it doesn't work correctly with older versions of Grafana. The Select component (which was its alternative) is now considered deprecated.

I think we should consider raising the minimum supported Grafana version. What do you think, @dmitryk-dk, @hagen1778?

As for me, to up the version is ok, but we need to test everything properly

We could have another option to use, use the old Combobox, but I am not sure it would work in the newest versions.

11.3.1 as @Loori-R mentioned, is not an old version and many users are using versions 10-11.

[Loori-R]

• There is no "old" Combobox - it was introduced as a new component and is currently marked as Alpha. View source

• The Select component is marked as Deprecated. View source

As an option, we could use Select for now since it’s more stable, and plan to migrate to Combobox in the future.
Guide: Migrating from Select to Combobox

[dmitryk-dk]

• There is no "old" Combobox - it was introduced as a new component and is currently marked as Alpha. View source
• The Select component is marked as Deprecated. View source

As an option, we could use Select for now since it’s more stable, and plan to migrate to Combobox in the future. Guide: Migrating from Select to Combobox

I think using Select instead of Combobox is a better idea. We do not need to change the supported version, and we do not need to use the Alpha component.

[archef2000]

@Loori-R Can comfirm that with the Select component everything works.

[Loori-R]

@Loori-R Can comfirm that with the Select component everything works.

@archef2000 Would you mind replacing Combobox with Select?

[archef2000]

Already on it

[Loori-R]

Already on it

It looks like there are still some Combobox components in the code:
https://github.com/VictoriaMetrics/victorialogs-datasource/pull/329/files#diff-35151418f87b2fad4cd2f8e9b4515cd9685e1c922d771a348463efc8672b15ecR12-R22

[archef2000]

I am not finished yet and will make a commit once all are changed and tested

[archef2000]

This query should give me the value types of ClientPort right?
/select/logsql/field_values?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&start=1751839200000&end=1752011999999&field=type

query: _msg:* | uniq by ClientPort | block_stats | uniq type
field: type

[Loori-R]

This query should give me the value types of ClientPort right? /select/logsql/field_values?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&start=1751839200000&end=1752011999999&field=type

query: _msg:* | uniq by ClientPort | block_stats | uniq type field: type

Not sure, but it seems like uniq by ClientPort and uniq type might overlap.
@dmitryk-dk, could you please help clarify?

[Loori-R]

Seems like the query won’t return ClientPort types — it just gets unique type values from logs matching the full query as a text filter, because /field_values doesn’t run the pipeline logic.

[dmitryk-dk]

_msg:* | uniq by ClientPort | block_stats | uniq type

Sorry guys missed you question

query _msg:* | uniq by ClientPort | block_stats will return

https://play-vmlogs.victoriametrics.com/select/vmui/#/?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats&g0.range_input=5m&g0.end_input=2025-07-09T11%3A56%3A01&g0.relative_time=last_5_minutes

field: ClientPorttype: constvalues_bytes: 0bloom_bytes: 0dict_items: 0dict_bytes: 0rows: 1part_path: inmemory

the next filter will get only type so the query like
_msg:* | uniq by ClientPort | block_stats | uniq type will return only type: const
https://play-vmlogs.victoriametrics.com/select/vmui/#/?query=_msg%3A*+%7C+uniq+by+ClientPort+%7C+block_stats+%7C+uniq+type&g0.range_input=5m&g0.end_input=2025-07-09T11%3A55%3A21&g0.relative_time=last_5_minutes

[archef2000]

I was in the assumption that /field_values will return the values of the field that are available after the query. Just noticed it in the value_type operation, but then I will have to change the logic for all. Should I change the getFieldList function with an additional type?

[Loori-R]

I don't quite understand what you're trying to achieve.

For the expression:
_msg:* | uniq by ClientPort | block_stats | uniq type

/fields_values can only be used for _msg (I'm not sure how usable it is for _msg, but still).
To substitute values for uniq, you need to use /field_names

[archef2000]

For the value_type operation I want to get the availabe value types of the field specified. The field_values is explained so that I get all field values of the specified field from the result of the query. The query gives me the correct result, but the endpoint seems to ignore the query.

[Loori-R]

Maybe we could just hardcode the supported value types (like const, dict, string, int64, float64, etc.) and allow users to enter a custom type if needed - that might be a simpler and more flexible solution.

[Loori-R]

I described just an example and asked to move all the handlers from the component to the handlers.

I would suggest extracting only complex logic - for example, if additional checks are needed or it takes more than 2-3 lines.

[archef2000]

Why are only filters respected in the field_values/field_names endpoints. When it is stated otherwise in the docs? @Loori-R @dmitryk-dk

[Loori-R]

Why are only filters respected in the field_values/field_names endpoints. When it is stated otherwise in the docs? @Loori-R @dmitryk-dk

The /field_values endpoint returns a list of unique values for a given field from the raw logs.
Just to clarify - could you provide an example of what you’re expecting to achieve with pipes in this context?

[archef2000]

For the every filter I construct all operation before that one and get all (at this point) available field values, names, types.

[Loori-R]

 Thanks for the pull request! LGTM!

[Loori-R]

@dmitryk-dk mind giving it a final check and merging?

[archef2000]

I would like the operation field names/values recommendation to be correct. So an answer to my question would be appreciated to know if I need a different approach or if the implementation will be adjusted according to the docs.

[Loori-R]

Why are only filters respected in the field_values/field_names endpoints. When it is stated otherwise in the docs? @Loori-R @dmitryk-dk

@valyala could you help clarify this?

[dmitryk-dk]

LGTM!

[archef2000]

There still is a problem when unselecting an item in all Editors using the MultiSelect component, as the values parameter to the onChange handler is empty.

[szibis]

Any ETA of releasing this datasource improvements for builder ??

[archef2000]

No as @valyala has not yet answered. The value prediction is completly wrong as the implementation does not match the official docs that I went after and there are some minor problems with some Editors that need fixing.

[kurayama]

@archef2000 is this related with this fix #308 (comment)?

[archef2000]

@kurayama Can confirm correctly working field_values/field_names endpoints.

[Loori-R]

@archef2000 could you please clarify if there are any critical issues in this PR?
If not, I’d suggest merging it (since it’s already quite large and has been open for a while), and then handling further improvements in separate issues/PRs.

[archef2000]

I am currently fixing some errors like every operation with the SortedFieldsEditor and testing every operation once in Grafana version 12 and 11.

[archef2000]

I just found some operations that are not implemented yet as they were just recently added.

[archef2000]

Should a version number be added to the Operations.tsx file?

[Loori-R]

Should a version number be added to the Operations.tsx file?

Yes, adding a version VictoriaLogs would help track changes.