Monitors VMware vSphere stats using govmomi. Sinks metrics to one of many time series backends.
Written in go to achieve fast sampling rates and high throughput sink. Successfully benchmarked against 3000 VM's, logging 150,000 metrics per minute to an ElasticSearch backend.
Compatible backends:
The dashboard example below is using the grafana UI. The backend is using ElasticSearch.
Define vSphere credentials and collection metrics in the JSON config file. An example configuration for the Contoso domain is found here.
Copy this config file to /etc/binaryname.json and modify as needed. As in windows "/etc" doesn't exist, the configuration file should be placed in the same directory as the vsphere-graphite binary.
Example:
cp vsphere-graphite-example.json /etc/vsphere-graphite.json
Metrics collection is performed by associating ObjectType groups with Metric groups. These are expressed via the vsphere scheme: group.metric.rollup
ObjectTypes are explained in this vSphere doc.
Performance metrics are explained in this vSphere doc.
You can select the extra data collected by using the "Properties" property:
- datastore: reports the datastores associated with a virtual machine
- host: reports the host the virtual machine runs on
- cluster: reports the cluster the virtual machine is in
- network: reports the network the virtual machine is connected to
- resourcepool: reports the resourcepool the virtual machine is in
- folder: reports the folder the virtual machine is in
- tags: reports the tags associated with the virtual machine
- numcpu: reports the number of virtual cpus the virtual machine has
- memorysizemb: reports the quantity of memory the virtual machine has
- disks: reports the logical disks capacity inside the virtual machine
- all: reports all the information
vCenter parameters can be set in the configuration file or via environment variable.
The configuration file needs the username, password and hostname of the vCenter (from sample config):
"VCenters": [
{ "Username": "CONTOSO\\Administrator", "Password": "$P@ssw0rd", "Hostname": "vc1.contoso.com" },
{ "Username": "CONTOSO\\Administrator", "Password": "$P@ssw0rd", "Hostname": "vc2.contoso.com" }
]If set via environment variable you can set multiple vcenters via VCENTER_*=<username>:<password>@<hostname>
To follow the example given in the sample file:
VCENTER_VC1=CONTOSO\\Administrator:$P@[email protected]
VCENTER_VC2=CONTOSO\\Administrator:$P@[email protected]
Backend parameters can be set in the config and will always be overridden by environment variables. This allows to use a generic config in a container image and set the backend by environment variables.
-
Type (CONFIG_TYPE):
Type of backend to use.
Currently "graphite", "influxdb", "thinfluxdb" (embeded influx client), "elastic", "prometheus", "thinprometheus" (embedded prometheus) "fluentd" and "kafka"
-
Hostname (CONFIG_HOSTNAME):
Hostname were the backend is running.
Prometheus will use this information to choose which interface (ip address) it listens to. An empty value means all interfaces.
i.e: 127.0.0.1 would limit to localhost.
-
Port (CONFIG_PORT):
Port to connect to for the backend.
Prometheus will use this to choose which port to listen to. By default it listens to port 9155.
-
Encrypted (CONFIG_ENCRYPTED):
Enable or disable TLS to the backend (true, false).
Only supported by "influx", "thininflux" and "elastic" backends.
Prometheus support for this would require certificate management.
-
Username (CONFIG_USERNAME):
Username to connect to the backend.
Only supported by "influx" and "thininflux" backends.
-
Password (CONFIG_PASSWORD):
Password to connect to the backend.
Only supported by "influx" and "thininflux" backends.
-
Database (CONFIG_DATABASE):
Database to use in the backend.
Only supported by "influx", "thininflux", "elastic" and "kafka" (kafka topic name)
-
NoArray (CONFIG_NOARRAY):
Multiple values metadata will be send as csv 'array' by default (NoArray is true by default). If set to false, only the first element is sent.
-
Prefix (CONFIG_PREFIX):
Prefix to use in the backend.
MOST OF THESE PARAMETERS SHOULDN'T BE TOUCHED
vCenter has limits on the quantity of data that can be returned in a request. The configuration following configuraiton file parameters can modify when to split permormance request into multiple requests:
- VCenterResultLimit: the limit of elements that can be returned in a request (defaults to 500000)
- VCenterInstanceRatio: a metric can return multiple results related to the object instances connected to the host or VM. (disks, network interfaces; the default ratio is 3.0)
All builds are pushed to docker:
Default tags includes:
- commit for specific commit in the branch (useful to run from tip)
- latest for latest release
- specific release tag or version
The JSON configuration file can be passed by mounting to /etc. Edit the configuration file and set it in the place you like here $(pwd)
docker run -t -v $(pwd)/vsphere-graphite.json:/etc/vsphere-graphite.json cblomart/vsphere-graphite:latest
Backend parameters can be set via environment variables to make docker user easier (having graphite or influx as another container).
A sample docker compose file is provided in the project. this sample will start:
- vcsim (vCenter simulator by govmomi)
- graphite ("Official" Graphite docker image) the port 80 will be published to access the web interface.
- vsphere-graphite with the necessary environment parameters to address the started backend and vcenter
To start this with swarm:
> docker stack deploy -v vsphere-graphite-graphite-test.yml vsphere-graphiteDid you know that you can run docker stack on a standalone host... so no need for docker-compose. Just:
> docker swarm init
Heavily based on govmomi but also on daemon which provides simple daemon/service integration.
Of course golang is needed.
refer to install and don't forget to set $GOPATH.
Gopath example:
mkdir /etc/golang
export GOPATH=/etc/golangThen install vsphere-graphite with GO:
go get github.com/cblomart/vsphere-graphite
The executable should be $GOPATH/bin/vsphere-graphite and is now a binary for your architecture/OS
vsphere-graphite
vsphere-graphite install
vsphere-graphite start
vsphere-graphite status
vsphere-graphite stop
vsphere-graphite remove
No open source projects would live and thrive without common effort. Here is the section were the ones that help are thanked:
Also keep in mind that if you can't contribute code, issues and improvement requests are also a key part of a project evolution! So don't hesitate and tell us what doesn't work or what you miss.
This project is largely alive because of the aforementioned contributors. Our time is precious bet it is even more precious to us when we can spend it on our beloved projects. So don't hesitate to make a donation (see badge)
The MIT License (MIT)
Copyright (c) 2016 cblomart
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
