Q&A - Socket source framing configurations

[alxhrck]

Question

Currently running v0.44.

The upgrade guide for v0.30 mentions max_length has been moved to the framing setting, but the documentation does not seem to indicate a configuration for using max_length with bytes framing method. Is it accurate that this setting cannot be configured?

I've noticed a behavior with events received through the source socket are treated as the new events (delimited) by the newline character. Changing framing.method: bytes. Does the Source Socket use one of the listed framing methods by default?

Vector Config

No response

Vector Logs

No response

[pront]

For TCP with the default bytes decoder, the default framing is newline-delimited. Changing framing.method to bytes stops splitting on newlines, but it also means Vector waits for the connection to close before emitting the buffered data as one event.

The bytes framer does not have a max_length setting. To limit newline-delimited TCP messages, use:

sources:
  incoming:
    type: socket
    mode: tcp
    address: 0.0.0.0:9000
    decoding:
      codec: bytes
    framing:
      method: newline_delimited
      newline_delimited:
        max_length: 16384

Messages over that limit are discarded. For a persistent TCP connection carrying multiple messages, choose framing that matches the sender's message boundaries, such as a delimiter or length prefix.

UDP is different: with the default decoder, each datagram is a frame, and the socket source still has a separate top-level max_length for incoming datagrams. The socket source reference covers these settings; the link in your question points to the sink.

Please don't hesitate to followup in this thread if you have further questions.