metrics suggestion: limits for ide, sata, scsi, virtio

[grin31]

subj

[znerol]

Hi, thanks for taking the time to fill PR #113. Would you mind explaining the use-case for scraping these limits? Unless I'm completely mistaken, this will scrape configuration values and not the actual runtime metrics.

[grin31]

@znerol hi there.
if parameter changed [GUI or CLI] on active VM the parameter applied without delay. you can change it on the fly.
for example:
qm set 533 -virtio1 ssd_pve351-x301:vm-533-disk-1,mbps=150,iops=15000

the main goal this PR is scrape limit settings and display line in IOPS and MB/s graphics

[znerol]

Thanks for taking the time to file this issue and the PR. I see how this could be useful.

Unfortunately scraping configuration has turned out to be inherently inefficient (see #55 and #58). Especially for big and growing deployments, this can get nasty quite quickly. In fact I intend to disable the config collector by default in an upcoming release (i.e., --no-collector.config).

PR #113 is using the exact same known-to-be-faulty mechanism to collect the desired data. To make matters worse, the problematic loop would be running twice after that PR landed and as a result the time to collect all metrics will double for many users.

All this considered, I think I have to reject the PR and the feature request. I am truly sorry for that.

[grin31]

@znerol thanks for reply. Collecting limits metrics is really useful for some Proxmox installation. Do you have any idea how to reduce data collection time?

[znerol]

I think this use-case could be solved using a proxmox hookscript in combination with either textfile collector or pushgateway

The hookscript would determine the limits and write them as metrics to the textfile directory or send it to the push gateway.

If the hookscript isn't practical as a trigger, other mechanisms could be used. E.g., a systemd.path unit watching /etc/pve/qemu-server/. If anything changes, then a script could extract the limits directly from the .conf files or alternatively use qm list and friends to get those values.

[grin31]

@znerol thank you for ideas.
if default disable gather this metrics
set
default=False
e.g.

parser.add_argument('--collector.iolimit', dest='collector_iolimit',
                        action=BooleanOptionalAction, default=False,
                        help='Exposes PVE io limit')

and make notion in README about slow works
will you merge it? )

[grin31]

confirm, on large installation scrape inefficient

....
# HELP pve_mbps_wr_max Proxmox vm config mbps_wr_max
# TYPE pve_mbps_wr_max gauge
pve_mbps_wr_max{device="virtio",device_name="virtio0",id="qemu/273",node="pve04-z601",type="qemu"} 30.0

real    0m6.366s
user    0m0.010s
sys     0m0.005s

[grin31]

@znerol
I have a proposition for you.
let's combine --collector.config and --collector.iolimit to save time and make these options
default=false
I can make PR.
What can you say about it?

[grin31]

@znerol
good day.
what about disable metric collection in default config?

[grin31]

@znerol good day.
Please see question below.

[grin31]

@elsbrock
Please see last question.