Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(outputs): Add inlong output plugin #16211

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/LICENSE_OF_DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ following works:
- github.com/antlr4-go/antlr [BSD 3-Clause "New" or "Revised" License](https://github.com/antlr/antlr4/blob/master/LICENSE.txt)
- github.com/apache/arrow-go [Apache License 2.0](https://github.com/apache/arrow-go/blob/main/LICENSE.txt)
- github.com/apache/arrow/go [Apache License 2.0](https://github.com/apache/arrow/blob/master/LICENSE.txt)
- github.com/apache/inlong/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang [Apache License 2.0](https://github.com/apache/inlong/blob/master/LICENSE)
- github.com/apache/iotdb-client-go [Apache License 2.0](https://github.com/apache/iotdb-client-go/blob/main/LICENSE)
- github.com/apache/thrift [Apache License 2.0](https://github.com/apache/thrift/blob/master/LICENSE)
- github.com/apapsch/go-jsonmerge [MIT License](https://github.com/apapsch/go-jsonmerge/blob/master/LICENSE)
Expand Down Expand Up @@ -99,6 +100,7 @@ following works:
- github.com/boschrexroth/ctrlx-datalayer-golang [MIT License](https://github.com/boschrexroth/ctrlx-datalayer-golang/blob/main/LICENSE)
- github.com/brutella/dnssd [MIT License](https://github.com/brutella/dnssd/blob/master/LICENSE)
- github.com/bufbuild/protocompile [Apache License 2.0](https://github.com/bufbuild/protocompile/blob/main/LICENSE)
- github.com/bwmarrin/snowflake [BSD 2-Clause "Simplified" License](https://github.com/bwmarrin/snowflake/blob/master/LICENSE)
- github.com/caio/go-tdigest [MIT License](https://github.com/caio/go-tdigest/blob/master/LICENSE)
- github.com/cenkalti/backoff [MIT License](https://github.com/cenkalti/backoff/blob/master/LICENSE)
- github.com/census-instrumentation/opencensus-proto [Apache License 2.0](https://github.com/census-instrumentation/opencensus-proto/blob/master/LICENSE)
Expand Down Expand Up @@ -161,6 +163,7 @@ following works:
- github.com/go-openapi/jsonreference [Apache License 2.0](https://github.com/go-openapi/jsonreference/blob/master/LICENSE)
- github.com/go-openapi/swag [Apache License 2.0](https://github.com/go-openapi/swag/blob/master/LICENSE)
- github.com/go-redis/redis [BSD 2-Clause "Simplified" License](https://github.com/go-redis/redis/blob/master/LICENSE)
- github.com/go-resty/resty [MIT License](https://github.com/go-resty/resty/blob/v2/LICENSE)
- github.com/go-sql-driver/mysql [Mozilla Public License 2.0](https://github.com/go-sql-driver/mysql/blob/master/LICENSE)
- github.com/go-stack/stack [MIT License](https://github.com/go-stack/stack/blob/master/LICENSE.md)
- github.com/go-stomp/stomp [Apache License 2.0](https://github.com/go-stomp/stomp/blob/master/LICENSE.txt)
Expand Down Expand Up @@ -315,7 +318,10 @@ following works:
- github.com/opencontainers/image-spec [Apache License 2.0](https://github.com/opencontainers/image-spec/blob/master/LICENSE)
- github.com/opensearch-project/opensearch-go [Apache License 2.0](https://github.com/opensearch-project/opensearch-go/blob/main/LICENSE.txt)
- github.com/opentracing/opentracing-go [Apache License 2.0](https://github.com/opentracing/opentracing-go/blob/master/LICENSE)
- github.com/oxtoacart/bpool [Apache License 2.0](https://github.com/oxtoacart/bpool/blob/master/LICENSE)
- github.com/p4lang/p4runtime [Apache License 2.0](https://github.com/p4lang/p4runtime/blob/main/LICENSE)
- github.com/panjf2000/ants [MIT License](https://github.com/panjf2000/ants/blob/dev/LICENSE)
- github.com/panjf2000/gnet [Apache License 2.0](https://github.com/panjf2000/gnet/blob/dev/LICENSE)
- github.com/paulmach/orb [MIT License](https://github.com/paulmach/orb/blob/master/LICENSE.md)
- github.com/pavlo-v-chernykh/keystore-go [MIT License](https://github.com/pavlo-v-chernykh/keystore-go/blob/main/LICENSE)
- github.com/pborman/ansi [BSD 3-Clause "New" or "Revised" License](https://github.com/pborman/ansi/blob/master/LICENSE)
Expand Down Expand Up @@ -388,6 +394,7 @@ following works:
- github.com/uber/jaeger-client-go [Apache License 2.0](https://github.com/jaegertracing/jaeger-client-go/blob/master/LICENSE)
- github.com/uber/jaeger-lib [Apache License 2.0](https://github.com/jaegertracing/jaeger-lib/blob/main/LICENSE)
- github.com/urfave/cli [MIT License](https://github.com/urfave/cli/blob/main/LICENSE)
- github.com/valyala/bytebufferpool [MIT License](https://github.com/valyala/bytebufferpool/blob/master/LICENSE)
- github.com/vapourismo/knx-go [MIT License](https://github.com/vapourismo/knx-go/blob/master/LICENSE)
- github.com/vishvananda/netlink [Apache License 2.0](https://github.com/vishvananda/netlink/blob/master/LICENSE)
- github.com/vishvananda/netns [Apache License 2.0](https://github.com/vishvananda/netns/blob/master/LICENSE)
Expand All @@ -406,6 +413,7 @@ following works:
- github.com/yuin/gopher-lua [MIT License](https://github.com/yuin/gopher-lua/blob/master/LICENSE)
- github.com/yusufpapurcu/wmi [MIT License](https://github.com/yusufpapurcu/wmi/blob/master/LICENSE)
- github.com/zeebo/xxh3 [BSD 2-Clause "Simplified" License](https://github.com/zeebo/xxh3/blob/master/LICENSE)
- github.com/zentures/cityhash [MIT License](https://github.com/zentures/cityhash/blob/master/LICENSE)
- go.mongodb.org/mongo-driver [Apache License 2.0](https://github.com/mongodb/mongo-go-driver/blob/master/LICENSE)
- go.opencensus.io [Apache License 2.0](https://github.com/census-instrumentation/opencensus-go/blob/master/LICENSE)
- go.opentelemetry.io/auto/sdk [Apache License 2.0](https://github.com/open-telemetry/opentelemetry-go-instrumentation/blob/main/sdk/LICENSE)
Expand Down Expand Up @@ -451,6 +459,7 @@ following works:
- gopkg.in/gorethink/gorethink.v3 [Apache License 2.0](https://github.com/rethinkdb/rethinkdb-go/blob/v3.0.5/LICENSE)
- gopkg.in/inf.v0 [BSD 3-Clause "New" or "Revised" License](https://github.com/go-inf/inf/blob/v0.9.1/LICENSE)
- gopkg.in/ini.v1 [Apache License 2.0](https://github.com/go-ini/ini/blob/master/LICENSE)
- gopkg.in/natefinch/lumberjack.v2 [MIT License](https://github.com/natefinch/lumberjack/blob/v2.0/LICENSE)
- gopkg.in/olivere/elastic.v5 [MIT License](https://github.com/olivere/elastic/blob/v5.0.76/LICENSE)
- gopkg.in/tomb.v1 [BSD 3-Clause Clear License](https://github.com/go-tomb/tomb/blob/v1/LICENSE)
- gopkg.in/tomb.v2 [BSD 3-Clause Clear License](https://github.com/go-tomb/tomb/blob/v2/LICENSE)
Expand Down
9 changes: 9 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ require (
github.com/antchfx/xmlquery v1.4.4
github.com/antchfx/xpath v1.3.3
github.com/apache/arrow-go/v18 v18.1.0
github.com/apache/inlong/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang v1.0.0-rc.3
github.com/apache/iotdb-client-go v1.3.3
github.com/apache/thrift v0.21.0
github.com/aristanetworks/goarista v0.0.0-20190325233358-a123909ec740
Expand Down Expand Up @@ -308,6 +309,7 @@ require (
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 // indirect
github.com/brutella/dnssd v1.2.14 // indirect
github.com/bufbuild/protocompile v0.10.0 // indirect
github.com/bwmarrin/snowflake v0.3.0 // indirect
github.com/caio/go-tdigest/v4 v4.0.1 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
Expand Down Expand Up @@ -350,6 +352,7 @@ require (
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.20.4 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-resty/resty/v2 v2.16.2 // indirect
github.com/go-stack/stack v1.8.1 // indirect
github.com/goburrow/modbus v0.1.0 // indirect
github.com/goburrow/serial v0.1.1-0.20211022031912-bfb69110f8dd // indirect
Expand Down Expand Up @@ -450,6 +453,9 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492 // indirect
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c // indirect
github.com/panjf2000/ants/v2 v2.10.0 // indirect
github.com/panjf2000/gnet/v2 v2.6.3 // indirect
github.com/paulmach/orb v0.11.1 // indirect
github.com/philhofer/fwd v1.1.3-0.20240612014219-fbbf4953d986 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
Expand Down Expand Up @@ -491,6 +497,7 @@ require (
github.com/twmb/murmur3 v1.1.7 // indirect
github.com/uber/jaeger-client-go v2.30.0+incompatible // indirect
github.com/uber/jaeger-lib v2.4.1+incompatible // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.2 // indirect
Expand All @@ -502,6 +509,7 @@ require (
github.com/yusufpapurcu/wmi v1.2.4 // indirect
github.com/zeebo/assert v1.3.1 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
github.com/zentures/cityhash v0.0.0-20131128155616-cdd6a94144ab // indirect
github.com/zitadel/logging v0.6.1 // indirect
github.com/zitadel/oidc/v3 v3.34.1 // indirect
github.com/zitadel/schema v1.3.0 // indirect
Expand Down Expand Up @@ -532,6 +540,7 @@ require (
gopkg.in/fsnotify.v1 v1.4.7 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/sourcemap.v1 v1.0.5 // indirect
gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
20 changes: 16 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -848,6 +848,8 @@ github.com/apache/arrow/go/v10 v10.0.1/go.mod h1:YvhnlEePVnBS4+0z3fhPfUy7W1Ikj0I
github.com/apache/arrow/go/v11 v11.0.0/go.mod h1:Eg5OsL5H+e299f7u5ssuXsuHQVEGC4xei5aX110hRiI=
github.com/apache/arrow/go/v15 v15.0.2 h1:60IliRbiyTWCWjERBCkO1W4Qun9svcYoZrSLcyOsMLE=
github.com/apache/arrow/go/v15 v15.0.2/go.mod h1:DGXsR3ajT524njufqf95822i+KTh+yea1jass9YXgjA=
github.com/apache/inlong/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang v1.0.0-rc.3 h1:tlnS+UUETsK32N2uY25wosc8NuNFXQAN9ppgYMnGcAM=
github.com/apache/inlong/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang v1.0.0-rc.3/go.mod h1:aqVmZ1f4b6XL61VeMyRwzr+P45ZvmyiFos9JtyzzJvs=
github.com/apache/iotdb-client-go v1.3.3 h1:qj1sr0trU8RITVtbdDBV/ZXeBZ8UnDyO8IIWPnOgano=
github.com/apache/iotdb-client-go v1.3.3/go.mod h1:3D6QYkqRmASS/4HsjU+U/3fscyc5M9xKRfywZsKuoZY=
github.com/apache/thrift v0.15.0/go.mod h1:PHK3hniurgQaNMZYaCLEqXKsYK8upmhPbmdP2FXSqgU=
Expand Down Expand Up @@ -978,6 +980,8 @@ github.com/buengese/sgzip v0.1.1 h1:ry+T8l1mlmiWEsDrH/YHZnCVWD2S3im1KLsyO+8ZmTU=
github.com/buengese/sgzip v0.1.1/go.mod h1:i5ZiXGF3fhV7gL1xaRRL1nDnmpNj0X061FQzOS8VMas=
github.com/bufbuild/protocompile v0.10.0 h1:+jW/wnLMLxaCEG8AX9lD0bQ5v9h1RUiMKOBOT5ll9dM=
github.com/bufbuild/protocompile v0.10.0/go.mod h1:G9qQIQo0xZ6Uyj6CMNz0saGmx2so+KONo8/KrELABiY=
github.com/bwmarrin/snowflake v0.3.0 h1:xm67bEhkKh6ij1790JB83OujPR5CzNe8QuQqAgISZN0=
github.com/bwmarrin/snowflake v0.3.0/go.mod h1:NdZxfVWX+oR6y2K0o6qAYv6gIOP9rjG0/E9WsDpxqwE=
github.com/caio/go-tdigest v3.1.0+incompatible h1:uoVMJ3Q5lXmVLCCqaMGHLBWnbGoN6Lpu7OAUPR60cds=
github.com/caio/go-tdigest v3.1.0+incompatible/go.mod h1:sHQM/ubZStBUmF1WbB8FAm8q9GjDajLC5T7ydxE3JHI=
github.com/caio/go-tdigest/v4 v4.0.1 h1:sx4ZxjmIEcLROUPs2j1BGe2WhOtHD6VSe6NNbBdKYh4=
Expand Down Expand Up @@ -1258,8 +1262,8 @@ github.com/go-redis/redis/v7 v7.4.1 h1:PASvf36gyUpr2zdOUS/9Zqc80GbM+9BDyiJSJDDOr
github.com/go-redis/redis/v7 v7.4.1/go.mod h1:JDNMw23GTyLNC4GZu9njt15ctBQVn7xjRfnwdHj/Dcg=
github.com/go-redis/redis/v8 v8.11.5 h1:AcZZR7igkdvfVmQTPnu9WE37LRrO/YrBH5zWyjDC0oI=
github.com/go-redis/redis/v8 v8.11.5/go.mod h1:gREzHqY1hg6oD9ngVRbLStwAWKhA0FEgq8Jd4h5lpwo=
github.com/go-resty/resty/v2 v2.13.1 h1:x+LHXBI2nMB1vqndymf26quycC4aggYJ7DECYbiz03g=
github.com/go-resty/resty/v2 v2.13.1/go.mod h1:GznXlLxkq6Nh4sU59rPmUw3VtgpO3aS96ORAI6Q7d+0=
github.com/go-resty/resty/v2 v2.16.2 h1:CpRqTjIzq/rweXUt9+GxzzQdlkqMdt8Lm/fuK/CAbAg=
github.com/go-resty/resty/v2 v2.16.2/go.mod h1:0fHAoK7JoBy/Ch36N8VFeMsK7xQOHhvWaC3iOktwmIU=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpvNJ1Y=
github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
Expand Down Expand Up @@ -2023,10 +2027,14 @@ github.com/openzipkin/zipkin-go v0.4.3 h1:9EGwpqkgnwdEIJ+Od7QVSEIH+ocmm5nPat0G7s
github.com/openzipkin/zipkin-go v0.4.3/go.mod h1:M9wCJZFWCo2RiY+o1eBCEMe0Dp2S5LDHcMZmk3RmK7c=
github.com/oracle/oci-go-sdk/v65 v65.69.2 h1:lROMJ8/VakGOGObAWUxTVY2AX1wQCUIzVqfL4Fb2Ay8=
github.com/oracle/oci-go-sdk/v65 v65.69.2/go.mod h1:IBEV9l1qBzUpo7zgGaRUhbB05BVfcDGYRFBCPlTcPp0=
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c h1:rp5dCmg/yLR3mgFuSOe4oEnDDmGLROTvMragMUXpTQw=
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c/go.mod h1:X07ZCGwUbLaax7L0S3Tw4hpejzu63ZrrQiUe6W0hcy0=
github.com/p4lang/p4runtime v1.4.0 h1:LbCCClz/5uJzLU+puL2aA/0Bz6xiZKxKVyVlTIhAWOQ=
github.com/p4lang/p4runtime v1.4.0/go.mod h1:OWAP4Wh9uKGnQjleslObpFE0REP78b5gR1pHyYmvNPQ=
github.com/panjf2000/ants/v2 v2.9.1 h1:Q5vh5xohbsZXGcD6hhszzGqB7jSSc2/CRr3QKIga8Kw=
github.com/panjf2000/ants/v2 v2.9.1/go.mod h1:7ZxyxsqE4vvW0M7LSD8aI3cKwgFhBHbxnlN8mDqHa1I=
github.com/panjf2000/ants/v2 v2.10.0 h1:zhRg1pQUtkyRiOFo2Sbqwjp0GfBNo9cUY2/Grpx1p+8=
github.com/panjf2000/ants/v2 v2.10.0/go.mod h1:7ZxyxsqE4vvW0M7LSD8aI3cKwgFhBHbxnlN8mDqHa1I=
github.com/panjf2000/gnet/v2 v2.6.3 h1:+Lw6FUAvgtc51bcs3w5bNQOsx8iVv5nEWVJUL4lhjsA=
github.com/panjf2000/gnet/v2 v2.6.3/go.mod h1:HpNv+iQrIOeil1eyhdnKDlui7jivyMf0K3xwaeHKnh8=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY=
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
Expand Down Expand Up @@ -2412,6 +2420,8 @@ github.com/zeebo/errs v1.3.0/go.mod h1:sgbWHsvVuTPHcqJJGQ1WhI5KbWlHYz+2+2C/LSEtC
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
github.com/zentures/cityhash v0.0.0-20131128155616-cdd6a94144ab h1:BD4YbH4Y0ysgbrP9jGuDB0BxkqyTRk6Y70o3D5Z5ayc=
github.com/zentures/cityhash v0.0.0-20131128155616-cdd6a94144ab/go.mod h1:SvJE1nX57VqPOyqkQGEGcJPWZqeB3FCZ8s7a0uSlG+A=
github.com/zitadel/logging v0.6.1 h1:Vyzk1rl9Kq9RCevcpX6ujUaTYFX43aa4LkvV1TvUk+Y=
github.com/zitadel/logging v0.6.1/go.mod h1:Y4CyAXHpl3Mig6JOszcV5Rqqsojj+3n7y2F591Mp/ow=
github.com/zitadel/oidc/v3 v3.34.1 h1:/rxx2HxEowd8Sdb8sxcRxTu9pLy3/TXBLrewKOUMTHA=
Expand Down Expand Up @@ -3346,6 +3356,8 @@ gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
gopkg.in/olivere/elastic.v5 v5.0.86 h1:xFy6qRCGAmo5Wjx96srho9BitLhZl2fcnpuidPwduXM=
gopkg.in/olivere/elastic.v5 v5.0.86/go.mod h1:M3WNlsF+WhYn7api4D87NIflwTV/c0iVs8cqfWhK+68=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
Expand Down
5 changes: 5 additions & 0 deletions plugins/outputs/all/inlong.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
//go:build !custom || outputs || outputs.inlong

package all

import _ "github.com/influxdata/telegraf/plugins/outputs/inlong" // register plugin
55 changes: 55 additions & 0 deletions plugins/outputs/inlong/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Inlong Output Plugin

This plugin writes telegraf metrics to
[Apache InLong](https://inlong.apache.org/docs/next/introduction).

## Global configuration options <!-- @/docs/includes/plugin_config.md -->

In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.

[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins

## Configuration

```toml @sample.conf
# Send telegraf metrics to Inlong
[[outputs.inlong]]
## It comes from the Inlong system, a `DataStreamGroup` (GroupID) aggregates multiple data streams into a business unit,
## enabling logical grouping, resource sharing, and permission isolation. Each group represents a cohesive business
## scenario (e.g., "order_processing_group").
group_id = "test_group" # (string) Unique identifier for the business unit

## It comes from the Inlong system, a `DataStream` (StreamID) defines a specific data pipeline with a unique source,
## format, and destination. It is part of a DataStreamGroup and operates within its business context.
stream_id = "test_stream" # (string) Unique identifier for the data stream within its group
## From the Inlong system, data streams group, it contains multiple data streams, and one Group represents
## one data business unit.
group_id = "test_group"

## It comes from the Inlong system, a `DataStream` (StreamID) defines a specific data pipeline with a unique source,
## format, and destination. It is part of a DataStreamGroup and operates within its business context.
stream_id = "test_stream" # (string) Unique identifier for the data stream within its group
## From the Inlong system, data stream, a stream has a specific data source, data format and data sink.
stream_id = "test_stream"

## The URL used to obtain the Inlong DataProxy IP list to which the data will be sent
manager_url = "http://127.0.0.1:8083"

## Data format to output.
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
data_format = "csv"

## The delimiter used when serializing data in CSV format needs to be consistent with the delimiter
## configured for Inlong, so that the data can be parsed properly after it reaches Inlong.
## It can be a space, vertical bar (|), comma (,), semicolon (;), asterisk (*), double quotes ("), etc.
csv_separator = "|"

## The final output field order here needs to be consistent with the field order defined by the data
## stream in Inlong
csv_columns = ["field.key","file.value"]
```
88 changes: 88 additions & 0 deletions plugins/outputs/inlong/inlong.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
package inlong

import (
"context"
_ "embed"
"fmt"

"github.com/apache/inlong/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy"

"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/internal"
"github.com/influxdata/telegraf/plugins/outputs"
)

//go:embed sample.conf
var sampleConfig string

const managerURLSuffix = "/inlong/manager/openapi/dataproxy/getIpList"

type Inlong struct {
GroupID string `toml:"group_id"`
StreamID string `toml:"stream_id"`
ManagerURL string `toml:"manager_url"`
Log telegraf.Logger `toml:"-"`

producerFunc func(groupId string, managerUrl string) (dataproxy.Client, error)
producer dataproxy.Client
serializer telegraf.Serializer
}

func (*Inlong) SampleConfig() string {
return sampleConfig
}

func (i *Inlong) SetSerializer(serializer telegraf.Serializer) {
i.serializer = serializer
}

func (i *Inlong) Connect() error {
producer, err := i.producerFunc(i.GroupID, i.ManagerURL+managerURLSuffix)
if err != nil {
return &internal.StartupError{Err: err, Retry: true}
}
i.producer = producer
return nil
}

func (i *Inlong) Close() error {
i.producer.Close()
return nil
}

func (i *Inlong) Write(metrics []telegraf.Metric) error {
for _, metric := range metrics {
b, err := i.serializer.Serialize(metric)
if err != nil {
return fmt.Errorf("could not serialize metric: %w", err)
}
err = i.producer.Send(context.Background(), dataproxy.Message{
GroupID: i.GroupID,
StreamID: i.StreamID,
Payload: b,
})
if err != nil {
return fmt.Errorf("could not send metric to GroupID %s StreamID %s: %w", i.GroupID, i.StreamID, err)
}
}
return nil
}

func newProducer(groupID, managerURL string) (dataproxy.Client, error) {
producer, err := dataproxy.NewClient(
dataproxy.WithGroupID(groupID),
dataproxy.WithURL(managerURL),
)
if err != nil {
return nil, fmt.Errorf("could not new producer for groupID %s managerURL %s: %w", groupID, managerURL, err)
}
return producer, nil
}

func init() {
outputs.Add("inlong", func() telegraf.Output {
return &Inlong{
producerFunc: newProducer,
}
})
}
Loading
Loading