Vector
Configuration file for Vector, an observability data pipeline agent and aggregator
| Type | object |
|---|---|
| File match |
vector.json
vector.toml
vector.yaml
|
| Schema URL | https://catalog.lintel.tools/schemas/schemastore/vector/latest.json |
| Source | https://www.schemastore.org/vector.json |
Validate with Lintel
npx @lintel/lintel checkA complete Vector configuration.
All of
Definitions
The user configuration to choose the metric tag strategy.
Options for building a LengthDelimitedDecoder or LengthDelimitedEncoder.
Length field byte order (little or big endian)
Number of bytes representing the field length
Number of bytes in the header before the length field
Maximum frame length
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Apache Avro serializer options.
The Avro schema definition.
Note: The following [apache_avro::types::Value] variants are not supported:
DateDecimalDurationFixedTimeMillis
For Avro datum encoded in Kafka messages, the bytes are prefixed with the schema ID. Set this to true to strip the schema ID prefix.
According to Confluent Kafka's document.
Config used to build a GelfDeserializer.
GELF-specific decoding options.
2 nested properties
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Configures the decoding validation mode.
GELF-specific decoding options.
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Configures the decoding validation mode.
Configures the decoding validation mode.
Config used to build a InfluxdbDeserializer.
Influxdb-specific decoding options.
1 nested properties
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Influxdb-specific decoding options.
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Config used to build a JsonDeserializer.
JSON-specific decoding options.
1 nested properties
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
JSON-specific decoding options.
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Config used to build a NativeJsonDeserializer.
Vector's native JSON-specific decoding options.
1 nested properties
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Vector's native JSON-specific decoding options.
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Config used to build an OtlpDeserializer.
The deserializer will try parsing in the order specified. This allows you to optimize
performance when you know the expected signal types. For example, if you only receive
traces, set this to ["traces"] to avoid attempting to parse as logs or metrics first.
If not specified, defaults to trying all types in order: logs, metrics, traces. Duplicate signal types are automatically removed while preserving order.
[
"logs",
"metrics",
"traces"
]OTLP signal type for prioritized parsing.
Config used to build a ProtobufDeserializer.
Protobuf-specific decoding options.
3 nested properties
A file path.
The name of the message type to use for serializing.
When enabled, the deserializer will output fields using their JSON names as defined
in the .proto file (e.g., jobDescription instead of job_description).
This is useful when working with data that needs to be converted to JSON or when interfacing with systems that use JSON naming conventions.
Protobuf-specific decoding options.
A file path.
The name of the message type to use for serializing.
When enabled, the deserializer will output fields using their JSON names as defined
in the .proto file (e.g., jobDescription instead of job_description).
This is useful when working with data that needs to be converted to JSON or when interfacing with systems that use JSON naming conventions.
Config used to build a SyslogDeserializer.
Syslog-specific decoding options.
1 nested properties
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Syslog-specific decoding options.
When true, invalid UTF-8 sequences are replaced with the U+FFFD REPLACEMENT CHARACTER.
Config used to build a VrlDeserializer.
VRL-specific decoding options.
2 nested properties
If not set, local is used.
VRL-specific decoding options.
If not set, local is used.
Config used to build a CharacterDelimitedDecoder.
Options for building a CharacterDelimitedDecoder.
2 nested properties
The character that delimits byte sequences.
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
Options for building a CharacterDelimitedDecoder.
The character that delimits byte sequences.
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
Config used to build a ChunkedGelfDecoder.
Options for building a ChunkedGelfDecoder.
4 nested properties
Decompression options for ChunkedGelfDecoder.
Note: A message can be composed of multiple chunks and this limit is applied to the whole message, not to individual chunks.
This limit takes only into account the message's payload and the GELF header bytes are excluded from the calculation. The message's payload is the concatenation of all the chunks' payloads.
The maximum number of pending incomplete messages. If this limit is reached, the decoder starts dropping chunks of new messages, ensuring the memory usage of the decoder's state is bounded. If this option is not set, the decoder does not limit the number of pending messages and the memory usage of its messages buffer can grow unbounded. This matches Graylog Server's behavior.
The timeout, in seconds, for a message to be fully received. If the timeout is reached, the decoder drops all the received chunks of the timed out message.
Options for building a ChunkedGelfDecoder.
Decompression options for ChunkedGelfDecoder.
Note: A message can be composed of multiple chunks and this limit is applied to the whole message, not to individual chunks.
This limit takes only into account the message's payload and the GELF header bytes are excluded from the calculation. The message's payload is the concatenation of all the chunks' payloads.
The maximum number of pending incomplete messages. If this limit is reached, the decoder starts dropping chunks of new messages, ensuring the memory usage of the decoder's state is bounded. If this option is not set, the decoder does not limit the number of pending messages and the memory usage of its messages buffer can grow unbounded. This matches Graylog Server's behavior.
The timeout, in seconds, for a message to be fully received. If the timeout is reached, the decoder drops all the received chunks of the timed out message.
Decompression options for ChunkedGelfDecoder.
Config used to build a LengthDelimitedDecoder.
Options for building a LengthDelimitedDecoder or LengthDelimitedEncoder.
4 nested properties
Length field byte order (little or big endian)
Number of bytes representing the field length
Number of bytes in the header before the length field
Maximum frame length
Config used to build a NewlineDelimitedDecoder.
Options for building a NewlineDelimitedDecoder.
1 nested properties
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
Options for building a NewlineDelimitedDecoder.
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
Config used to build a OctetCountingDecoder.
Options for building a OctetCountingDecoder.
1 nested properties
The maximum length of the byte buffer.
Options for building a OctetCountingDecoder.
The maximum length of the byte buffer.
Config used to build a VarintLengthDelimitedDecoder.
Maximum frame length
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
Encoding configuration.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
Apache Avro serializer options.
The Avro schema.
Config used to build a CefSerializer.
Config used to build a CefSerializer.
8 nested properties
Unique identifier for each event type. Identifies the type of event reported. The value length must be less than or equal to 1023.
Identifies the product of a vendor. The part of a unique device identifier. No two products can use the same combination of device vendor and device product. The value length must be less than or equal to 63.
Identifies the vendor of the product. The part of a unique device identifier. No two products can use the same combination of device vendor and device product. The value length must be less than or equal to 63.
Identifies the version of the problem. The combination of the device product, vendor, and this value make up the unique id of the device that sends messages. The value length must be less than or equal to 31.
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
CEF version.
The collection of key-value pairs. Keys are the keys of the extensions, and values are paths that point to the extension values of a log event. The event can have any number of key-value pairs in any order.
Config used to build a CefSerializer.
Unique identifier for each event type. Identifies the type of event reported. The value length must be less than or equal to 1023.
Identifies the product of a vendor. The part of a unique device identifier. No two products can use the same combination of device vendor and device product. The value length must be less than or equal to 63.
Identifies the vendor of the product. The part of a unique device identifier. No two products can use the same combination of device vendor and device product. The value length must be less than or equal to 63.
Identifies the version of the problem. The combination of the device product, vendor, and this value make up the unique id of the device that sends messages. The value length must be less than or equal to 31.
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
CEF version.
The collection of key-value pairs. Keys are the keys of the extensions, and values are paths that point to the extension values of a log event. The event can have any number of key-value pairs in any order.
CEF version.
Config used to build a CsvSerializer.
Config used to build a CsvSerializer.
7 nested properties
If a field is not present in the event, the output for that field is an empty string.
Values of type Array, Object, and Regex are not supported, and the
output for any of these types is an empty string.
Sets the capacity (in bytes) of the internal buffer used in the CSV writer. This defaults to 8192 bytes (8KB).
The field delimiter to use when writing CSV.
This is enabled by default, but you can disable it. When disabled, quotes in field data are escaped instead of doubled.
In some variants of CSV, quotes are escaped using a special escape character like \ (instead of escaping quotes by doubling them).
To use this, double_quotes needs to be disabled as well; otherwise, this setting is ignored.
The quote character to use when writing CSV.
The user configuration to choose the metric tag strategy.
Config used to build a CsvSerializer.
If a field is not present in the event, the output for that field is an empty string.
Values of type Array, Object, and Regex are not supported, and the
output for any of these types is an empty string.
Sets the capacity (in bytes) of the internal buffer used in the CSV writer. This defaults to 8192 bytes (8KB).
The field delimiter to use when writing CSV.
This is enabled by default, but you can disable it. When disabled, quotes in field data are escaped instead of doubled.
In some variants of CSV, quotes are escaped using a special escape character like \ (instead of escaping quotes by doubling them).
To use this, double_quotes needs to be disabled as well; otherwise, this setting is ignored.
The quote character to use when writing CSV.
The user configuration to choose the metric tag strategy.
The user configuration to choose the metric tag strategy.
Config used to build a GelfSerializer.
Config used to build a GelfSerializer.
1 nested properties
Maximum size for each GELF chunked datagram (including 12-byte header).
Chunking starts when datagrams exceed this size.
For Graylog target, keep at or below 8192 bytes; for Vector target (gelf decoding with chunked_gelf framing), up to 65,500 bytes is recommended.
Config used to build a GelfSerializer.
Maximum size for each GELF chunked datagram (including 12-byte header).
Chunking starts when datagrams exceed this size.
For Graylog target, keep at or below 8192 bytes; for Vector target (gelf decoding with chunked_gelf framing), up to 65,500 bytes is recommended.
Config used to build a JsonSerializer.
Options for the JsonSerializer.
1 nested properties
Whether to use pretty JSON formatting.
The user configuration to choose the metric tag strategy.
Options for the JsonSerializer.
Whether to use pretty JSON formatting.
Config used to build a ProtobufSerializer.
Protobuf serializer options.
3 nested properties
A file path.
The name of the message type to use for serializing.
When enabled, the serializer looks for fields using their JSON names as defined
in the .proto file (for example jobDescription instead of job_description).
This is useful when working with data that has already been converted from JSON or when interfacing with systems that use JSON naming conventions.
Protobuf serializer options.
A file path.
The name of the message type to use for serializing.
When enabled, the serializer looks for fields using their JSON names as defined
in the .proto file (for example jobDescription instead of job_description).
This is useful when working with data that has already been converted from JSON or when interfacing with systems that use JSON naming conventions.
The syslog RFC standard to use for formatting.
Config used to build a SyslogSerializer.
Syslog serializer options.
{
"rfc": "rfc5424",
"facility": null,
"severity": null,
"app_name": null,
"proc_id": null,
"msg_id": null
}6 nested properties
If not provided, the encoder checks for a semantic "service" field. If that is also missing, it defaults to "vector".
Path to a field in the event to use for the facility. Defaults to "user".
Path to a field in the event to use for the msg ID.
Path to a field in the event to use for the proc ID.
The syslog RFC standard to use for formatting.
Path to a field in the event to use for the severity. Defaults to "informational".
Syslog serializer options.
If not provided, the encoder checks for a semantic "service" field. If that is also missing, it defaults to "vector".
Path to a field in the event to use for the facility. Defaults to "user".
Path to a field in the event to use for the msg ID.
Path to a field in the event to use for the proc ID.
The syslog RFC standard to use for formatting.
Path to a field in the event to use for the severity. Defaults to "informational".
Config used to build a TextSerializer.
The user configuration to choose the metric tag strategy.
Config used to build a CharacterDelimitedEncoder.
Configuration for character-delimited framing.
1 nested properties
The ASCII (7-bit) character that delimits byte sequences.
Configuration for character-delimited framing.
The ASCII (7-bit) character that delimits byte sequences.
Framing configuration.
Config used to build a LengthDelimitedEncoder.
Options for building a LengthDelimitedDecoder or LengthDelimitedEncoder.
4 nested properties
Length field byte order (little or big endian)
Number of bytes representing the field length
Number of bytes in the header before the length field
Maximum frame length
Config used to build a VarintLengthDelimitedEncoder.
Maximum frame length
Serializer configuration.
The format in which a timestamp should be represented.
Transformations to prepare an event for serialization.
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Framing configuration.
The format in which a timestamp should be represented.
A file path.
Configuration of the region/endpoint to use when interacting with an AWS service.
Use the HTTP authentication with HTTPS only. The authentication credentials are passed as an HTTP header without any additional encryption beyond what is provided by the transport itself.
Many methods exist for matching events, such as using a VRL expression, a Datadog Search query string, or hard-coded matchers like "must be a metric" or "fields A, B, and C must match these constraints".
As VRL is the most common way to apply conditions to events, this type provides a shortcut to define VRL expressions directly in the configuration by passing the VRL expression as a string:
condition = '.message == "hooray"'
When other condition types are required, they can be specified with an enum-style notation:
condition.type = 'datadog_search'
condition.source = 'NOT "foo"'
HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an HTTP header without any additional encryption beyond what is provided by the transport itself.
Configuration for SASL authentication when interacting with Kafka.
Configuration of the authentication strategy when interacting with NATS.
Configuration settings for InfluxDB v0.x/v1.x.
Configuration settings for InfluxDB v2.x.
All compression algorithms use the default compression level unless otherwise specified.
Options for determining the health of an endpoint.
Content encoding.
Configuration of multi-line aggregation.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
Unsigned integer template.
Wrapper for sensitive strings containing credentials
A metric.
Tags for a metric series.
List of allowed origin IP networks. IP addresses must be in CIDR notation.
TCP keepalive settings for socket-based components.
TLS configuration.
Configures the TLS options for incoming/outgoing connections.
TlsEnableableConfig for sources, adding metadata from the client certificate.
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
A wrapper around OwnedValuePath that allows it to be used in Vector config.
This requires a valid path to be used. If you want to allow optional paths,
use [optional_path::OptionalValuePath].
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
This can refer to any valid timezone as defined in the TZ database, or "local" which refers to the system local timezone. It will default to the globally configured timezone.
File position to use when reading a new file.
HTTP response status code
A span of time, in fractional seconds.
A span of time, in whole milliseconds.
A span of time, in whole seconds.
A file path.
An internet socket address, either IPv4 or IPv6.
AMQP connection options.
The URI has the format of
amqp://<user>:<password>@<host>:<port>/<vhost>?timeout=<seconds>.
The default vhost can be specified by using a value of %2f.
To connect over TLS, a scheme of amqps can be specified instead. For example,
amqps://.... Additional TLS settings, such as client certificate verification, can be
configured under the tls section.
Configuration of the authentication strategy for interacting with AWS services.
IMDS Client Configuration for authenticating with AWS.
Configuration of the region/endpoint to use when interacting with an AWS service.
Custom endpoint for use with AWS-compatible services.
Use the HTTP authentication with HTTPS only. The authentication credentials are passed as an HTTP header without any additional encryption beyond what is provided by the transport itself.
Shared MQTT configuration for sources and sinks.
MQTT server address (The broker’s domain name or IP address).
MQTT client ID.
Connection keep-alive interval.
Maximum packet size
MQTT password.
TCP port of the MQTT server to connect to.
TLS configuration.
MQTT username.
Shared websocket configuration for sources and sinks.
This should include the protocol and host, but can also include the port, path, and any other valid part of a URI.
Note: Using the wss:// protocol requires enabling tls.
HTTP Authentication.
If this option is not configured, pings are not sent on an interval.
If the ping_timeout is not set, pings are still sent but there is no expectation of pong
response times.
If a response is not received within this time, the connection is re-established.
TLS configuration.
Many methods exist for matching events, such as using a VRL expression, a Datadog Search query string, or hard-coded matchers like "must be a metric" or "fields A, B, and C must match these constraints".
As VRL is the most common way to apply conditions to events, this type provides a shortcut to define VRL expressions directly in the configuration by passing the VRL expression as a string:
condition = '.message == "hooray"'
When other condition types are required, they can be specified with an enum-style notation:
condition.type = 'datadog_search'
condition.source = 'NOT "foo"'
Many methods exist for matching events, such as using a VRL expression, a Datadog Search query string, or hard-coded matchers like "must be a metric" or "fields A, B, and C must match these constraints".
They can specified with an enum-style notation:
condition.type = 'datadog_search'
condition.source = 'NOT "foo"'
A condition that uses the Datadog Search query syntax against an event.
The query string.
A condition that uses the Vector Remap Language (VRL) boolean expression against an event.
The VRL boolean expression.
The runtime to use for executing VRL code.
An input describes not only the type of event to insert, but also which transform within the configuration to insert it to.
Component identifier.
Only relevant when type is log.
Only relevant when type is metric.
Only relevant when type is vrl.
Can be either raw, vrl, log, or `metric.
Use this only when the input event should be a raw event (i.e. unprocessed/undecoded log
event) and when the input type is set to raw.
Configure output for component when generated with graph command
They are added to the node as provided
Fully resolved sink component.
Fully resolved source component.
Fully resolved transform component.
Configuration for the unit_test sink.
Name of the test that this sink is being used for.
List of names of the transform/branch associated with this sink.
Configuration for the unit_test source.
Configuration for the unit_test_stream sink.
Configuration for the unit_test_stream source.
Configuration for the file enrichment table.
File-specific settings.
2 nested properties
File encoding configuration.
A file path.
This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.
Timestamp coercions need to be prefaced with timestamp|, for example "timestamp|%F". Timestamp specifiers can use either of the following:
- One of the built-in-formats listed in the
Timestamp Formatstable below. - The time format specifiers from Rust’s
chronolibrary.
Types
boolstringfloatintegerdatetimestamp(see the table below for formats)
Timestamp Formats
| Format | Description | Example |
|---|---|---|
%F %T | YYYY-MM-DD HH:MM:SS | 2020-12-01 02:37:54 |
%v %T | DD-Mmm-YYYY HH:MM:SS | 01-Dec-2020 02:37:54 |
%FT%T | ISO 8601/RFC 3339, without time zone | 2020-12-01T02:37:54 |
%FT%TZ | ISO 8601/RFC 3339, UTC | 2020-12-01T09:37:54Z |
%+ | ISO 8601/RFC 3339, UTC, with time zone | 2020-12-01T02:37:54-07:00 |
%a, %d %b %Y %T | RFC 822/RFC 2822, without time zone | Tue, 01 Dec 2020 02:37:54 |
%a %b %e %T %Y | ctime format | Tue Dec 1 02:37:54 2020 |
%s | UNIX timestamp | 1606790274 |
%a %d %b %T %Y | date command, without time zone | Tue 01 Dec 02:37:54 2020 |
%a %d %b %T %Z %Y | date command, with time zone | Tue 01 Dec 02:37:54 PST 2020 |
%a %d %b %T %z %Y | date command, with numeric time zone | Tue 01 Dec 02:37:54 -0700 2020 |
%a %d %b %T %#z %Y | date command, with numeric time zone (minutes can be missing or present) | Tue 01 Dec 02:37:54 -07 2020 |
{}Configuration for the geoip enrichment table.
A file path.
MaxMind includes localized versions of some of the fields within their database, such as country name. This setting can control which of those localized versions are returned by the transform.
More information on which portions of the geolocation data are localized, and what languages are available, can be found here.
Configuration for the memory enrichment table.
By default, all writes are made visible immediately.
Configuration of internal metrics
{
"include_key_tag": false
}1 nested properties
This is useful for distinguishing between different keys while monitoring. However, the tag's cardinality is unbounded.
The namespace to use for logs. This overrides the global setting.
By default, there is no size limit.
The scan interval used to look for expired records. This is provided as an optimization to ensure that TTL is updated, but without doing too many cache scans.
Configuration for source functionality.
TTL (time-to-live in seconds) is used to limit the lifetime of data stored in the cache. When TTL expires, data behind a specific key in the cache is removed. TTL is reset when the key is replaced.
An optional path that deserializes an empty string to None.
Configuration for the mmdb enrichment table.
A file path.
Configuration of the authentication strategy for interacting with GCP services.
Either an API key or a path to a service account credentials JSON file can be specified.
If both are unset, the GOOGLE_APPLICATION_CREDENTIALS environment variable is checked for a filename. If no
filename is named, an attempt is made to fetch an instance service account for the compute instance the program is
running on. If this is not on a GCE instance, then you must define it with an API key or service account
credentials JSON file.
Either an API key or a path to a service account credentials JSON file can be specified.
If both are unset, the GOOGLE_APPLICATION_CREDENTIALS environment variable is checked for a filename. If no
filename is named, an attempt is made to fetch an instance service account for the compute instance the program is
running on. If this is not on a GCE instance, then you must define it with an API key or service account
credentials JSON file.
Skip all authentication handling. For use with integration tests only.
HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an HTTP header without any additional encryption beyond what is provided by the transport itself.
Configuration of HTTP server keepalive parameters.
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The type of a query parameter's value, determines if it's treated as a plain string or a VRL expression.
Represents a query parameter value, which can be a simple string or a typed object indicating whether the value is a string or a VRL expression.
Configuration of the query parameter value for HTTP requests.
Configuration of internal metrics for file-based components.
This is useful for distinguishing between different files while monitoring. However, the tag's cardinality is unbounded.
Kafka authentication configuration.
Configuration for SASL authentication when interacting with Kafka.
Only PLAIN- and SCRAM-based mechanisms are supported when configuring SASL authentication using sasl.*. For
other mechanisms, librdkafka_options.* must be used directly to configure other librdkafka-specific values.
If using sasl.kerberos.* as an example, where * is service.name, principal, kinit.md, etc., then
librdkafka_options.* as a result becomes librdkafka_options.sasl.kerberos.service.name,
librdkafka_options.sasl.kerberos.principal, etc.
See the librdkafka documentation for details.
SASL authentication is not supported on Windows.
The SASL mechanism to use.
The SASL password.
The SASL username.
Mode of operation of the line aggregator.
Configuration of the authentication strategy when interacting with NATS.
Credentials file configuration.
Path to credentials file.
NKeys configuration.
Conceptually, this is equivalent to a public key.
Conceptually, this is equivalent to a private key.
Token configuration.
Wrapper for sensitive strings containing credentials
Username and password configuration.
Wrapper for sensitive strings containing credentials
Username.
Configuration for the http provider.
Configuration for the aws_secrets_manager secrets backend.
Configuration for the directory secrets backend.
A file path.
Remove trailing whitespace from file contents.
Configuration for the exec secrets backend.
The path to the script or binary must be the first argument.
Configuration for the command that will be execed
The timeout, in seconds, to wait for the command to complete.
Configuration for the command that will be execed
Configuration for the file secrets backend.
A file path.
Configuration for the test secrets backend.
Fixed value to replace all secrets with.
A value which can be (de)serialized from one or many instances of T.
Configurable sinks in Vector.
Supports AMQP version 0.9.1
Configuration for the appsignal sink.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
All compression algorithms use the default compression level unless otherwise specified.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
The URI for the AppSignal API to send data to.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the aws_cloudwatch_logs sink.
Configuration for the aws_cloudwatch_metrics sink.
Base configuration for the aws_kinesis_ sinks.
The actual specific sink configuration types should either wrap this in a newtype wrapper,
or should extend it in a new struct with serde(flatten).
Configuration for the aws_kinesis_firehose sink.
Configuration for the aws_kinesis_streams sink.
Configuration for the aws_s3 sink.
Base Configuration aws_s_s for sns and sqs sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Configuration of the authentication strategy for interacting with AWS services.
This value is a template which should result in a unique string for each event. See the AWS documentation for more about how AWS does message deduplication.
Can be applied only to FIFO queues.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the aws_sns sink.
Configuration for the aws_sqs sink.
Configuration for the axiom sink.
Configuration for the azure_blob sink.
Configuration for the azure_logs_ingestion sink.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Configuration of the authentication strategy for interacting with Azure services.
{
"azure_tenant_id": "",
"azure_client_id": "",
"azure_client_secret": ""
}Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
The setting of log_schema.timestamp_key, usually timestamp, is used as the source.
Most schemas use TimeGenerated, but some use Timestamp (legacy) or EventStartTime (ASIM) std_columns.
Configuration for the azure_monitor_logs sink.
Can only contain letters, numbers, and underscores (_), and may not exceed 100 characters.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
The setting of log_schema.timestamp_key, usually timestamp, is used here by default.
This field should be used in rare cases where TimeGenerated should point to a specific log
field. For example, use this field to set the log field source_timestamp as holding the
value that should be used as TimeGenerated on the Azure side.
Configuration for the blackhole sink.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
By default, there is no limit.
Configuration for the clickhouse sink.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}When specified, events are encoded together as a single batch.
This is mutually exclusive with per-event encoding based on the format field.
All compression algorithms use the default compression level unless otherwise specified.
The database that contains the table that data is inserted into.
Sets date_time_input_format to best_effort, allowing ClickHouse to properly parse RFC3339/ISO 8601.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
The format to parse input data.
Sets insert_distributed_one_random_shard, allowing ClickHouse to insert data into a random shard when using Distributed Table Engine.
Query settings for the clickhouse sink.
{
"async_insert_settings": {
"enabled": null,
"wait_for_processing": null,
"wait_for_processing_timeout": null,
"deduplicate": null,
"max_data_size": null,
"max_query_number": null
}
}1 nested properties
Async insert-related settings.
{
"enabled": null,
"wait_for_processing": null,
"wait_for_processing_timeout": null,
"deduplicate": null,
"max_data_size": null,
"max_query_number": null
}6 nested properties
If left unspecified, use the default provided by the ClickHouse server.
If left unspecified, use the default provided by the ClickHouse server.
If left unspecified, use the default provided by the ClickHouse server.
If left unspecified, use the default provided by the ClickHouse server.
If left unspecified, use the default provided by the ClickHouse server.
If left unspecified, use the default provided by the ClickHouse server.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
If left unspecified, use the default provided by the ClickHouse server.
Configuration for the console sink.
Configuration for the databend sink.
The table that data is inserted into.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
The username and password to authenticate with. Overrides the username and password in DSN.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Compression configuration.
The database that contains the table that data is inserted into. Overrides the database in DSN.
Configures how events are encoded into raw bytes.
{
"codec": "json",
"json": {
"pretty": false
}
}Defines how missing fields are handled for NDJson. Refer to https://docs.databend.com/sql/sql-reference/file-format-options#null_field_as
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
The TLS configuration to use when connecting to the Databend server.
Shared configuration for Datadog sinks. Contains the maximum set of common settings that applies to all DD sink components.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
If an event has a Datadog API key set explicitly in its metadata, it takes precedence over this setting.
This value can also be set by specifying the DD_API_KEY environment variable.
The value specified here takes precedence over the environment variable.
The endpoint must contain an HTTP scheme, and may specify a hostname or IP address and port. The API path should NOT be specified as this is handled by the sink.
If set, overrides the site option.
This value can also be set by specifying the DD_SITE environment variable.
The value specified here takes precedence over the environment variable.
If not specified by the environment variable, a default value of
datadoghq.com is taken.
Configuration for the datadog_events sink.
Configuration for the datadog_logs sink.
Configuration for the datadog_metrics sink.
Configuration for the datadog_traces sink.
Configuration for the doris sink.
Configuration for the elasticsearch sink.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Amazon OpenSearch Serverless requires this option to be set to auto (the default).
Elasticsearch Authentication strategies.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Elasticsearch bulk mode configuration.
{
"action": "index",
"index": "vector-%Y.%m.%d",
"template_fallback_index": null,
"version": null,
"version_type": "internal"
}5 nested properties
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
The default index to write events to if the template in bulk.index cannot be resolved
Version field value.
Possible values are internal, external or external_gt and external_gte.
All compression algorithms use the default compression level unless otherwise specified.
Elasticsearch data stream mode configuration.
This is only relevant for Elasticsearch <= 6.X. If you are using >= 7.0 you do not need to set this option since Elasticsearch has removed it.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
The endpoint must contain an HTTP scheme, and may specify a hostname or IP address and port.
The endpoint must contain an HTTP scheme, and may specify a
hostname or IP address and port.
The endpoint may include basic authentication credentials,
e.g., <https://user:[email protected]>. If credentials are provided in the endpoint,
they will be used to authenticate against Elasticsearch.
If auth is specified and the endpoint contains credentials,
a configuration error will be raised.
[]By default, the _id field is not set, which allows Elasticsearch to set this
automatically. Setting your own Elasticsearch IDs can hinder performance.
Configuration for the metric_to_log transform.
Elasticsearch Indexing mode.
Amazon OpenSearch service type
The name of the pipeline to apply.
Custom parameters to add to the query string for each HTTP request sent to Elasticsearch.
Outbound HTTP request settings.
To avoid duplicates in Elasticsearch, please use option id_key.
The type field was deprecated in Elasticsearch 7.x and removed in Elasticsearch 8.x.
If enabled, the doc_type option is ignored.
Configuration for the file sink.
Configuration for the gcp_cloud_storage sink.
Configuration for the gcp_pubsub sink.
Configuration for the gcp_stackdriver_logs sink.
Configuration for the gcp_stackdriver_metrics sink.
Configuration for the gcp_chronicle_unstructured sink.
Configuration for the greptimedb_logs sink.
The endpoint of the GreptimeDB server.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}All compression algorithms use the default compression level unless otherwise specified.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Custom headers to add to the HTTP request sent to GreptimeDB. Note that these headers will override the existing headers.
Custom parameters to add to the query string for each HTTP request sent to GreptimeDB.
This is required if your instance has authentication enabled.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
Pipeline version to be used for the logs.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
This is required if your instance has authentication enabled.
Configuration items for GreptimeDB
This sink uses GreptimeDB's gRPC interface for data ingestion. By default, GreptimeDB listens to port 4001 for gRPC protocol.
The address must include a port.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Default to public, the default database of GreptimeDB.
Database can be created via create database statement on
GreptimeDB. If you are using GreptimeCloud, use dbname from the
connection information of your instance.
Set gRPC compression encoding for the request
Default to none, gzip or zstd is supported.
This is to keep consistency with GreptimeDB's naming pattern. By
default, this sink will use val for value column name, and ts for
time index name. When turned on, greptime_value and
greptime_timestamp will be used for these names.
If you are using this Vector sink together with other data ingestion sources of GreptimeDB, like Prometheus Remote Write and Influxdb Line Protocol, it is highly recommended to turn on this.
Also if there is a tag name conflict from your data source, for
example, you have a tag named as val or ts, you need to turn on
this option to avoid the conflict.
Default to false for compatibility.
This is required if your instance has authentication enabled.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
This is required if your instance has authentication enabled.
Configuration items for GreptimeDB
This sink uses GreptimeDB's gRPC interface for data ingestion. By default, GreptimeDB listens to port 4001 for gRPC protocol.
The address must include a port.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Default to public, the default database of GreptimeDB.
Database can be created via create database statement on
GreptimeDB. If you are using GreptimeCloud, use dbname from the
connection information of your instance.
Set gRPC compression encoding for the request
Default to none, gzip or zstd is supported.
This is to keep consistency with GreptimeDB's naming pattern. By
default, this sink will use val for value column name, and ts for
time index name. When turned on, greptime_value and
greptime_timestamp will be used for these names.
If you are using this Vector sink together with other data ingestion sources of GreptimeDB, like Prometheus Remote Write and Influxdb Line Protocol, it is highly recommended to turn on this.
Also if there is a tag name conflict from your data source, for
example, you have a tag named as val or ts, you need to turn on
this option to avoid the conflict.
Default to false for compatibility.
This is required if your instance has authentication enabled.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
This is required if your instance has authentication enabled.
Configuration for the honeycomb sink.
Wrapper for sensitive strings containing credentials
The dataset to which logs are sent.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
All compression algorithms use the default compression level unless otherwise specified.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Honeycomb's endpoint to send logs to
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
A subset of the HTTP methods described in RFC 9110, section 9.1 are supported.
Configuration for the http sink.
Configuration for the humio_logs sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}All compression algorithms use the default compression level unless otherwise specified.
The scheme (http or https) must be specified. No path should be included since the paths defined
by the Splunk API are used.
If unset, Humio defaults it to none.
An optional path that deserializes an empty string to None.
In public-facing APIs, this must (if present) be equal to the repository used to create the ingest token used for authentication.
In private cluster setups, Humio can be configured to allow these to be different.
For more information, see Humio’s Format of Data.
Can be used to tag events by specifying fields starting with #.
For more information, see Humio’s Format of Data.
[]Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Typically the filename the logs originated from. Maps to @source in Humio.
An optional path that deserializes an empty string to None.
Overrides the name of the log field used to retrieve the nanosecond-enabled timestamp to send to Humio.
Configuration for the humio_metrics sink.
Configuration settings for InfluxDB v0.x/v1.x.
Only relevant when using InfluxDB v0.x/v1.x.
Only relevant when using InfluxDB v0.x/v1.x.
Only relevant when using InfluxDB v0.x/v1.x.
Only relevant when using InfluxDB v0.x/v1.x.
Only relevant when using InfluxDB v0.x/v1.x.
Configuration settings for InfluxDB v2.x.
Only relevant when using InfluxDB v2.x and above.
Only relevant when using InfluxDB v2.x and above.
Wrapper for sensitive strings containing credentials
Configuration for the influxdb_logs sink.
Configuration for the influxdb_metrics sink.
Configuration for the kafka sink.
Configuration for the keep sink.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Keeps endpoint to send logs to
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the loki sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
All compression algorithms use the default compression level unless otherwise specified.
Both keys and values are templateable, which enables you to attach dynamic labels to events.
Valid label keys include *, and prefixes ending with *, to allow for the expansion of
objects into multiple labels. See Label expansion for more information.
Note: If the set of labels has high cardinality, this can cause drastic performance issues with Loki. To prevent this from happening, reduce the number of unique label keys and values.
Some sources may generate events with timestamps that aren't in chronological order. Even though the sink sorts the events before sending them to Loki, there is a chance that another event could come in that is out of order with the latest events sent to Loki. Prior to Loki 2.4.0, this was not supported and would result in an error during the push request.
If you're using Loki 2.4.0 or newer, Accept is the preferred action, which lets Loki handle
any necessary sorting/reordering. If you're using an earlier version, then you must use Drop
or RewriteTimestamp depending on which option makes the most sense for your use case.
The path to use in the URL of the Loki instance.
Whether or not to delete fields from the event when they are used as labels.
Whether or not to delete fields from the event when they are used in structured metadata.
The timestamp is still sent as event metadata for Loki to use for indexing.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Both keys and values are templateable, which enables you to attach dynamic structured metadata to events.
Valid metadata keys include *, and prefixes ending with *, to allow for the expansion of
objects into multiple metadata entries. This follows the same logic as Label expansion.
{}When running Loki locally, a tenant ID is not required.
Configuration for the mezmo (formerly logdna) sink.
Wrapper for sensitive strings containing credentials
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}The default app that is set for events that do not contain a file or app field.
The default environment that is set for events that do not contain an env field.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
The IP address that is attached to each batch of events.
The MAC address that is attached to each batch of events.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
The tags that are attached to each batch of events.
Configuration for the mezmo (formerly logdna) sink.
Wrapper for sensitive strings containing credentials
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}The default app that is set for events that do not contain a file or app field.
The default environment that is set for events that do not contain an env field.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
The IP address that is attached to each batch of events.
The MAC address that is attached to each batch of events.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
The tags that are attached to each batch of events.
Configuration for the mqtt sink
Configuration for the nats sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
The URL must take the form of nats://server:port.
If the port is not specified it defaults to 4222.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
If set, the subject must belong to an existing JetStream stream.
{
"enabled": false,
"headers": null
}2 nested properties
Whether to enable Jetstream.
A map of NATS headers to be included in each message.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
{
"timeout_secs": 60,
"rate_limit_duration_secs": 1,
"rate_limit_num": 9223372036854775807,
"retry_attempts": 9223372036854775807,
"retry_max_duration_secs": 30,
"retry_initial_backoff_secs": 1,
"retry_jitter_mode": "Full",
"adaptive_concurrency": {
"initial_concurrency": 1,
"decrease_ratio": 0.9,
"ewma_alpha": 0.4,
"rtt_deviation_scale": 2.5,
"max_concurrency_limit": 200
}
}9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the new_relic sink.
Wrapper for sensitive strings containing credentials
New Relic API endpoint.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
All compression algorithms use the default compression level unless otherwise specified.
Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
New Relic region.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the OpenTelemetry sink.
Protocol configuration
Configuration for the papertrail sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
Configures the send buffer size using the SO_SNDBUF option on the socket.
Configuration for the postgres sink.
The PostgreSQL server connection string. It can contain the username and password. See PostgreSQL documentation about connection strings for more information about valid formats and options that can be used.
The table that data is inserted into. This table parameter is vulnerable to SQL injection attacks as Vector does not validate or sanitize it, you must not use untrusted input. This parameter will be directly interpolated in the SQL query statement, as table names as parameters in prepared statements are not allowed in PostgreSQL.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Note that as PostgreSQL's jsonb_populate_recordset function is used to insert events,
a single event in the batch can make the whole batch to fail. For example, if a single event within the batch triggers
a unique constraint violation in the destination table, the whole event batch will fail.
As a workaround, triggers on constraint violations
can be defined at a database level to change the behavior of the insert operation on specific tables.
Alternatively, setting max_events batch setting to 1 will make each event to be inserted independently,
so events that trigger a constraint violation will not affect the rest of the events.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}The postgres connection pool size. See this for more information about why a connection pool should be used.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the prometheus_exporter sink.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
An internet socket address, either IPv4 or IPv6.
[
0.005,
0.01,
0.025,
0.05,
0.1,
0.25,
0.5,
1.0,
2.5,
5.0,
10.0
]This namespace is only used if a metric has no existing namespace. When a namespace is
present, it is used as a prefix to the metric name, and separated with an underscore (_).
It should follow the Prometheus naming conventions.
While distributions as a lossless way to represent a set of samples for a metric is supported, Prometheus clients (the application being scraped, which is this sink) must aggregate locally into either an aggregated histogram or aggregated summary.
[
0.5,
0.75,
0.9,
0.95,
0.99
]This can sometimes be useful when the source of metrics leads to their timestamps being too far in the past for Prometheus to allow them, such as when aggregating metrics over long time periods, or when replaying old metrics from a disk buffer.
Configuration for the prometheus_remote_write sink.
The endpoint should include the scheme and the path to write to.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Authentication strategies.
The batch config for remote write.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null,
"aggregate": true
}[
0.005,
0.01,
0.025,
0.05,
0.1,
0.25,
0.5,
1.0,
2.5,
5.0,
10.0
]All compression algorithms use the default compression level unless otherwise specified.
This namespace is only used if a metric has no existing namespace. When a namespace is
present, it is used as a prefix to the metric name, and separated with an underscore (_).
It should follow the Prometheus naming conventions.
If unset, sending unique incremental metrics to this sink will cause indefinite memory growth.
[
0.5,
0.75,
0.9,
0.95,
0.99
]Outbound HTTP request settings for the Prometheus remote write sink.
{
"timeout_secs": 60,
"rate_limit_duration_secs": 1,
"rate_limit_num": 9223372036854775807,
"retry_attempts": 9223372036854775807,
"retry_max_duration_secs": 30,
"retry_initial_backoff_secs": 1,
"retry_jitter_mode": "Full",
"adaptive_concurrency": {
"initial_concurrency": 1,
"decrease_ratio": 0.9,
"ewma_alpha": 0.4,
"rtt_deviation_scale": 2.5,
"max_concurrency_limit": 200
},
"headers": {}
}If set, a header named X-Scope-OrgID is added to outgoing requests with the value of this setting.
This may be used by Cortex or other remote services to identify the tenant making the request.
Configuration for the pulsar sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
The endpoint should specify the pulsar protocol and port.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Authentication configuration.
Event batching behavior.
{
"max_events": null,
"max_bytes": null
}2 nested properties
The maximum size of a batch before it is flushed.
Note this is an unsigned 32 bit integer which is a smaller capacity than many of the other sink batch settings.
Supported compression types for Pulsar.
Custom connection retry options configuration for the Pulsar client.
If the field does not exist in the log event or metric tags, a blank value will be used.
If omitted, the key is not sent.
Pulsar uses a hash of the key to choose the topic-partition or uses round-robin if the record has no key.
The name of the producer. If not specified, the default name assigned by Pulsar is used.
If omitted, no properties will be written.
TLS options configuration for the Pulsar client.
Configuration for the redis sink.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
The URL must take the form of protocol://server:port/db where the protocol can either be
redis or rediss for connections secured via TLS.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Redis data type to store messages in.
List-specific options.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
{
"timeout_secs": 60,
"rate_limit_duration_secs": 1,
"rate_limit_num": 9223372036854775807,
"retry_attempts": 9223372036854775807,
"retry_max_duration_secs": 30,
"retry_initial_backoff_secs": 1,
"retry_jitter_mode": "Full",
"adaptive_concurrency": {
"initial_concurrency": 1,
"decrease_ratio": 0.9,
"ewma_alpha": 0.4,
"rtt_deviation_scale": 2.5,
"max_concurrency_limit": 200
}
}9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Controls how Redis Sentinel will connect to the servers belonging to it.
If this is specified, endpoint will be used to reach sentinel instances instead of a
redis instance.
Sorted Set-specific options
The Sematext region to send data to.
Configuration for the sematext_logs sink.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}Transformations to prepare an event for serialization.
3 nested properties
List of fields that are excluded from the encoded event.
List of fields that are included in the encoded event.
Format used for timestamp fields.
Setting this option overrides the region option.
The Sematext region to send data to.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the sematext_metrics sink.
This namespace is only used if a metric has no existing namespace. When a namespace is
present, it is used as a prefix to the metric name, and separated with a period (.).
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Setting this option overrides the region option.
The Sematext region to send data to.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for the socket sink.
Unix Domain Socket configuration.
Splunk HEC acknowledgement configuration.
Configuration for the splunk_hec_logs sink.
Wrapper for sensitive strings containing credentials
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
The scheme (http or https) must be specified. No path should be included since the paths defined
by the Splunk API are used.
Splunk HEC acknowledgement configuration.
{
"indexer_acknowledgements_enabled": true,
"query_interval": 10,
"retry_limit": 30,
"max_pending_acks": 1000000
}This option is only relevant to Splunk v8.x and above, and is only applied when
endpoint_target is set to event.
Setting this to true causes Splunk to extract the timestamp from the message text
rather than use the timestamp embedded in the event. The timestamp must be in the format
yyyy-mm-dd hh:mm:ss.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}All compression algorithms use the default compression level unless otherwise specified.
Splunk HEC endpoint configuration.
By default, the global log_schema.host_key option is used if log
events are Legacy namespaced, or the semantic meaning of "host" is used, if defined.
If not specified, the default index defined within Splunk is used.
[]Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
This is typically the filename the logs originated from.
If unset, the Splunk collector sets it.
If unset, Splunk defaults to httpevent.
By default, either the global log_schema.timestamp_key option is used
if log events are Legacy namespaced, or the semantic meaning of "timestamp" is used, if defined.
Configuration of the splunk_hec_metrics sink.
Wrapper for sensitive strings containing credentials
The scheme (http or https) must be specified. No path should be included since the paths defined
by the Splunk API are used.
Splunk HEC acknowledgement configuration.
{
"indexer_acknowledgements_enabled": true,
"query_interval": 10,
"retry_limit": 30,
"max_pending_acks": 1000000
}Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}All compression algorithms use the default compression level unless otherwise specified.
This namespace is only used if a metric has no existing namespace. When a namespace is
present, it is used as a prefix to the metric name, and separated with a period (.).
An optional path that deserializes an empty string to None.
If not specified, the default index defined within Splunk is used.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
This is typically the filename the logs originated from.
If unset, the Splunk collector sets it.
If unset, Splunk defaults to httpevent.
Configuration for the statsd sink.
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Event batching behavior.
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Event batching behavior.
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Event batching behavior.
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
Event batching behavior.
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
All compression algorithms use the default compression level unless otherwise specified.
Compression level.
Outbound HTTP request settings.
The jitter mode to use for retry backoff behavior.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
Options for determining the health of an endpoint.
Initial delay between attempts to reactivate endpoints once they become unhealthy.
Both IP addresses and hostnames/fully qualified domain names (FQDNs) are accepted formats.
The address must include a port.
A Unix Domain Socket sink.
A file path.
Configuration for the vector sink.
Both IP address and hostname are accepted formats.
The address must include a port.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Event batching behavior.
{
"max_bytes": null,
"max_events": null,
"timeout_secs": null
}3 nested properties
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
The maximum size of a batch before it is flushed.
The maximum age of a batch before it is flushed.
If set to true, requests are compressed with gzip.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
9 nested properties
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
5 nested properties
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly
when latency increases.
Note: The new limit is rounded down after applying this ratio.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
Datadog recommends setting this value to your service's average limit if you're seeing that it takes a
long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the
adaptive_concurrency_limit metric.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
The time window used for the rate_limit_num option.
The maximum number of requests allowed within the rate_limit_duration_secs time window.
The maximum number of retries to make for failed requests.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
The jitter mode to use for retry backoff behavior.
The maximum amount of time to wait between retries.
Datadog highly recommends that you do not lower this value below the service's internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
Version of the configuration.
Configuration for the webhdfs sink.
Configuration for the websocket sink.
Configuration for the websocket_server sink.
An internet socket address, either IPv4 or IPv6.
Encoding configuration. Configures how events are encoded into raw bytes. The selected encoding also determines which input types (logs, metrics, traces) are supported.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
Configuration of internal metrics
{}1 nested properties
Extra tags to add to all metrics generated by this component
{}Configuration for message buffering which enables message replay for clients that connect later.
Configuration of websocket subprotocol handling.
{
"type": "specific",
"supported_subprotocols": []
}Configurable sources in Vector.
Supports AMQP version 0.9.1
Configuration for the apache_metrics source.
The list of mod_status endpoints to scrape metrics from.
Disabled if empty.
Configuration for the aws_ecs_metrics source.
If empty, the URI is automatically discovered based on the latest version detected.
By default:
- The version 4 endpoint base URI is stored in the environment variable
ECS_CONTAINER_METADATA_URI_V4. - The version 3 endpoint base URI is stored in the environment variable
ECS_CONTAINER_METADATA_URI. - The version 2 endpoint base URI is
169.254.170.2/v2/.
Disabled if empty.
If empty, the version is automatically discovered based on environment variables.
By default:
- Version 4 is used if the environment variable
ECS_CONTAINER_METADATA_URI_V4is defined. - Version 3 is used if the environment variable
ECS_CONTAINER_METADATA_URI_V4is not defined, but the environment variableECS_CONTAINER_METADATA_URIis defined. - Version 2 is used if neither of the environment variables
ECS_CONTAINER_METADATA_URI_V4orECS_CONTAINER_METADATA_URIare defined.
Configuration for the aws_kinesis_firehose source.
An internet socket address, either IPv4 or IPv6.
If set to true, when incoming requests contains an access key sent by AWS Firehose, it is kept in the
event secrets as "aws_kinesis_firehose_access_key".
AWS Kinesis Firehose can be configured to pass along a user-configurable access key with each request. If
configured, access_key should be set to the same value. Otherwise, all requests are allowed.
AWS Kinesis Firehose can be configured to pass along a user-configurable access key with each request. If
configured, access_keys should be set to the same value. Otherwise, all requests are allowed.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global setting.
Some services, like AWS CloudWatch Logs, compresses the events with gzip, before sending them AWS Kinesis Firehose. This option can be used to automatically decompress them before forwarding them to the next component.
Note that this is different from Content encoding option of the Firehose HTTP endpoint destination. That option controls the content encoding of the entire HTTP request.
Configuration for the aws_s3 source.
Configuration for the aws_sqs source.
Configuration for the datadog_agent source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
If this is set to true, logs are not accepted by the component.
If this is set to true, metrics (beta) are not accepted by the component.
If this is set to true, traces (alpha) are not accepted by the component.
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global setting.
For a source component named agent, the received logs, metrics (beta), and traces (alpha) can then be
configured as input to other components by specifying agent.logs, agent.metrics, and
agent.traces, respectively.
If this is set to true, when log events contain the field ddtags, the string value that
contains a list of key:value pairs set by the Agent is parsed and expanded into an array.
If not set, responses to completed requests will block indefinitely until connected
transforms or sinks are ready to receive the events. When this happens, the sending Datadog
Agent will eventually time out the request and drop the connection, resulting Vector
generating an "Events dropped." error and incrementing the component_discarded_events_total
internal metric. By setting this option to a value less than the Agent's timeout, Vector
will instead respond to the Agent with a HTTP 503 Service Unavailable error, emit a warning,
and increment the component_timed_out_events_total internal metric instead.
If this is set to true, metric names are split at the first '.' into a namespace and name.
For example, system.cpu.usage would be split into namespace system and name cpu.usage.
If false, the full metric name is used without splitting. This may be useful if you are using a
default namespace for metrics in sinks connected to this source.
If this is set to true, when incoming events contain a Datadog API key, it is
stored in the event metadata and used if the event is sent to a Datadog sink.
Configuration for the demo_logs source.
Configuration for the dnstap source.
Configuration for the docker_logs source.
Enables automatic merging of partial events.
Use an HTTPS URL to enable TLS encryption.
If absent, the DOCKER_HOST environment variable is used. If DOCKER_HOST is also absent,
the default Docker local socket (/var/run/docker.sock on Unix platforms,
//./pipe/docker_engine on Windows) is used.
Matching is prefix first, so specifying a value of foo would match any container named foo as well as any
container whose name started with foo. This applies equally whether matching container IDs or names.
By default, the source collects logs for all containers. If exclude_containers is configured, any
container that matches a configured exclusion is excluded even if it is also included with
include_containers, so care should be taken when using prefix matches as they cannot be overridden by a
corresponding entry in include_containers, for example, excluding foo by attempting to include foo-specific-id.
This can be used in conjunction with include_containers.
By default, the global log_schema.host_key option is used.
Matching is prefix first, so specifying a value of foo would match any container named foo as well as any
container whose name started with foo. This applies equally whether matching container IDs or names.
By default, the source collects logs for all containers. If include_containers is configured, only
containers that match a configured inclusion and are also not excluded get matched.
This can be used in conjunction with exclude_containers.
If not provided, all images are included.
Labels should follow the syntax described in the Docker object labels documentation.
The namespace to use for logs. This overrides the global setting.
If not specified, multiline aggregation is disabled.
If auto_partial_merge is disabled, partial events are emitted with a log field, set by this
configuration value, indicating that the event is not complete.
Only relevant when connecting to Docker with an HTTPS URL.
If not configured, the environment variable DOCKER_CERT_PATH is used. If DOCKER_CERT_PATH is absent, then DOCKER_CONFIG is used. If both environment variables are absent, the certificates in ~/.docker/ are read.
Configuration for the eventstoredb_metrics source.
By default, eventstoredb is used.
Endpoint to scrape stats from.
Configuration for the exec source.
The command to run, plus any arguments required.
Mode of operation for running the command.
Whether or not to clear the environment before setting custom environment variables.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Custom environment variables to set or update when running the command. If a variable name already exists in the environment, its value is replaced.
Whether or not the output from stderr should be included when generating events.
The namespace to use for logs. This overrides the global setting.
The maximum buffer size allowed before a log event is generated.
Configuration options for scheduled commands.
Configuration options for streaming commands.
The directory in which to run the command.
Configuration for the file source.
Array of file patterns to include. Globbing is supported.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
By default, the global data_dir option is used.
Make sure the running user has write permissions to this directory.
If this directory is specified, then Vector will attempt to create it.
Character set encoding.
Takes precedence over the include option. Note: The exclude patterns are applied after the attempt to glob everything
in include. This means that all files are first matched by include and then filtered by the exclude
patterns. This can be impactful if include contains directories with contents that are not accessible.
[]An optional path that deserializes an empty string to None.
This is important for checkpointing when file rotation is used.
{
"strategy": "checksum",
"ignored_header_bytes": 0,
"lines": 1
}A span of time, in whole milliseconds.
By default, the global log_schema.host_key option is used.
Set to "" to suppress this key.
Checkpoints are still written normally.
This may be useful when used with source directories containing dangling symlinks.
Ignore files with a data modification date older than the specified number of seconds.
Configuration of internal metrics for file-based components.
1 nested properties
This is useful for distinguishing between different files while monitoring. However, the tag's cardinality is unbounded.
String sequence used to separate one file line from another.
The namespace to use for logs. This overrides the global setting.
This protects against malformed lines or tailing incorrect files.
This allows distributing the reads more or less evenly across the files.
String value used to identify the start of a multi-line message.
How long to wait for more data when aggregating a multi-line message, in milliseconds.
If not specified, multiline aggregation is disabled.
The value is the byte offset of the start of the line within the file.
Off by default, the offset is only added to the event if this is set.
Instead of balancing read capacity fairly across all watched files, prioritize draining the oldest files before moving on to read data from more recent files.
File position to use when reading a new file.
If not specified, files are not removed.
Whether or not to start reading from the beginning of a new file.
Configuration for the file_descriptor source.
The file descriptor number to read from.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
By default, the global host_key option is used.
The namespace to use for logs. This overrides the global setting.
Messages larger than this are truncated.
Configuration for the stdin source.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
By default, the global log_schema.host_key option is used.
The namespace to use for logs. This overrides the global setting.
Messages larger than this are truncated.
Configuration for the fluent source.
Configuration for the gcp_pubsub source.
Configuration for heroku_logs source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global setting.
Accepts the wildcard (*) character for query parameters matching a specified pattern.
Specifying "*" results in all query parameters included in the log event.
These override any values included in the body with conflicting names.
[]Filtering configuration.
The patterns are matched using globbing.
The patterns are matched using globbing.
Configuration for the host_metrics source.
This collector is only available on Linux systems, and only supports either version 2 or hybrid cgroups.
{
"levels": 100,
"base": null,
"groups": {
"includes": null,
"excludes": null
}
}Defaults to all collectors.
[
"cpu",
"disk",
"filesystem",
"load",
"host",
"memory",
"network",
"process",
"cgroups",
"tcp"
]Options for the disk metrics collector.
{
"devices": {
"includes": null,
"excludes": null
}
}1 nested properties
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
Options for the filesystem metrics collector.
{
"devices": {
"includes": null,
"excludes": null
},
"filesystems": {
"includes": null,
"excludes": null
},
"mountpoints": {
"includes": null,
"excludes": null
}
}3 nested properties
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
Overrides the default namespace for the metrics emitted by the source.
Options for the network metrics collector.
{
"devices": {
"includes": null,
"excludes": null
}
}1 nested properties
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
Options for the process metrics collector.
{
"processes": {
"includes": null,
"excludes": null
}
}1 nested properties
Filtering configuration.
2 nested properties
The patterns are matched using globbing.
The patterns are matched using globbing.
?matches any single character.*matches any (possibly empty) sequence of characters.**matches the current directory and arbitrary subdirectories. This sequence must form a single path component, so both**aandb**are invalid and will result in an error. A sequence of more than two consecutive*characters is also invalid.[...]matches any character inside the brackets. Character sequences can also specify ranges of characters, as ordered by Unicode, so e.g.[0-9]specifies any character between 0 and 9 inclusive. An unclosed bracket is invalid.[!...]is the negation of[...], i.e. it matches any characters not in the brackets.
The metacharacters ?, *, [, ] can be matched by using brackets (e.g. [?]). When a ] occurs immediately
following [ or [! then it is interpreted as being part of, rather then ending, the character set, so ] and NOT
] can be matched by []] and [!]] respectively. The - character can be specified inside a character sequence
pattern by placing it at the start or the end, e.g. [abc-].
Configuration for the http_client source.
The full path must be specified.
HTTP Authentication.
Can be a static string or a VRL expression.
When a body is provided, the Content-Type header is automatically set to
application/json unless explicitly overridden in the headers configuration.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
One or more values for the same header can be provided.
{}The namespace to use for logs. This overrides the global setting.
HTTP method.
One or more values for the same parameter key can be provided.
The parameters provided in this option are appended to any parameters
manually provided in the endpoint option.
VRL functions are supported within query parameters. You can
use functions like now() to dynamically modify query
parameter values.
{}A span of time, in fractional seconds.
TLS configuration.
Configuration for the http_server source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
For json and ndjson encodings, the fields of the JSON objects are output as separate fields.
Accepts the wildcard (*) character for headers matching a specified pattern.
Specifying "*" results in all headers included in the log event.
These headers are not included in the JSON payload if a field with a conflicting name exists.
[]An optional path that deserializes an empty string to None.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global setting.
HTTP method.
The URL path on which log event POST requests are sent.
An optional path that deserializes an empty string to None.
Accepts the wildcard (*) character for query parameters matching a specified pattern.
Specifying "*" results in all query parameters included in the log event.
These override any values included in the body with conflicting names.
[]If set to true, only requests using the exact URL path specified in path are accepted. Otherwise,
requests sent to a URL path that starts with the value of path are accepted.
With strict_path set to false and path set to "", the configured HTTP source accepts requests from
any URL path.
Configuration for the http_server source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
For json and ndjson encodings, the fields of the JSON objects are output as separate fields.
Accepts the wildcard (*) character for headers matching a specified pattern.
Specifying "*" results in all headers included in the log event.
These headers are not included in the JSON payload if a field with a conflicting name exists.
[]An optional path that deserializes an empty string to None.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global setting.
HTTP method.
The URL path on which log event POST requests are sent.
An optional path that deserializes an empty string to None.
Accepts the wildcard (*) character for query parameters matching a specified pattern.
Specifying "*" results in all query parameters included in the log event.
These override any values included in the body with conflicting names.
[]If set to true, only requests using the exact URL path specified in path are accepted. Otherwise,
requests sent to a URL path that starts with the value of path are accepted.
With strict_path set to false and path set to "", the configured HTTP source accepts requests from
any URL path.
Configuration for the internal_logs source.
By default, the global log_schema.host_key option is used.
Set to "" to suppress this key.
The namespace to use for logs. This overrides the global setting.
An optional path that deserializes an empty string to None.
Configuration for the internal_metrics source.
Overrides the default namespace for the metrics emitted by the source.
A span of time, in fractional seconds.
Tag configuration for the internal_metrics source.
{
"host_key": null,
"pid_key": null
}2 nested properties
The value is the peer host's address, including the port. For example, 1.2.3.4:9000.
By default, the global log_schema.host_key option is used.
Set to "" to suppress this key.
By default, this is not set and the tag is not automatically added.
Configuration for the journald source.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
This option limits the size of the batch.
Only include entries that occurred after the current boot of the system.
By default, the global data_dir option is used.
Make sure the running user has write permissions to this directory.
If this directory is specified, then Vector will attempt to create it.
If exclude_units is specified, it is merged into this list.
{}Unit names lacking a . have .service appended to make them a valid service unit
name.
[]If specified, it is merged to the command line arguments as-is.
[]If empty or not present, all journal fields are accepted.
If include_units is specified, it is merged into this list.
{}If empty or not present, all units are accepted.
Unit names lacking a . have .service appended to make them a valid service unit name.
Note: This option matches only the _SYSTEMD_UNIT field, which is narrower than journalctl --unit.
Messages from systemd about unit lifecycle (start/stop) have _SYSTEMD_UNIT=init.scope and will not match.
To capture these, explicitly include init.scope or use include_matches for finer control.
[]If not set, journalctl uses the default system journal path.
This value is passed to journalctl through the --namespace option.
If not set, journalctl uses the default namespace.
If not set, a search is done for the journalctl path.
The namespace to use for logs. This overrides the global setting.
Has no effect unless the value of the field is already an integer.
Only include entries that appended to the journal after the entries have been read.
Configuration for the kafka source.
Configuration for the kubernetes_logs source.
Partial events are messages that were split by the Kubernetes Container Runtime log driver.
By default, the global data_dir option is used.
Make sure the running user has write permissions to this directory.
If this directory is specified, then Vector will attempt to create it.
A list of glob patterns to exclude from reading the files.
The built-in Node filter uses self_node_name to only watch Pods located on the same Node.
If your files share a common header that is not always a fixed size,
If the file has less than this amount of lines, it won't be read at all.
A span of time, in whole milliseconds.
Ignore files with a data modification date older than the specified number of seconds.
A list of glob patterns to include while reading the files.
This is useful to compute the latency between important event processing
stages. For example, the time delta between when a log line was written and when it was
processed by the kubernetes_logs source.
Setting to false prevents Vector from pulling in namespaces and thus namespace label fields will not
be available. This helps reduce load on the kube-apiserver and lowers daemonset memory usage in clusters
with many namespaces.
Configuration of internal metrics for file-based components.
1 nested properties
This is useful for distinguishing between different files while monitoring. However, the tag's cardinality is unbounded.
If not set, a connection to Kubernetes is made using the in-cluster configuration.
The namespace to use for logs. This overrides the global setting.
This protects against malformed lines or tailing incorrect files.
This protects against malformed lines or tailing incorrect files.
Note that, if auto_partial_merge is false, this config will be ignored. Also, if max_line_bytes is too small to reach the continuation character, then this
config will have no practical impact (the same is true of auto_partial_merge). Finally, the smaller of max_merged_line_bytes and max_line_bytes will apply
if auto_partial_merge is true, so if this is set to be 1 MiB, for example, but max_line_bytes is set to ~2.5 MiB, then every line greater than 1 MiB will be dropped.
This allows distributing the reads more or less evenly across the files.
Configuration for how the events are enriched with Namespace metadata.
{
"namespace_labels": ".kubernetes.namespace_labels"
}1 nested properties
An optional path that deserializes an empty string to None.
Configuration for how the events are enriched with Node metadata.
{
"node_labels": ".kubernetes.node_labels"
}1 nested properties
An optional path that deserializes an empty string to None.
Instead of balancing read capacity fairly across all watched files, prioritize draining the oldest files before moving on to read data from more recent files.
Configuration for how the events are enriched with Pod metadata.
{
"pod_name": ".kubernetes.pod_name",
"pod_namespace": ".kubernetes.pod_namespace",
"pod_uid": ".kubernetes.pod_uid",
"pod_ip": ".kubernetes.pod_ip",
"pod_ips": ".kubernetes.pod_ips",
"pod_labels": ".kubernetes.pod_labels",
"pod_annotations": ".kubernetes.pod_annotations",
"pod_node_name": ".kubernetes.pod_node_name",
"pod_owner": ".kubernetes.pod_owner",
"container_name": ".kubernetes.container_name",
"container_id": ".kubernetes.container_id",
"container_image": ".kubernetes.container_image",
"container_image_id": ".kubernetes.container_image_id"
}13 nested properties
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
File position to use when reading a new file.
Configured to use an environment variable by default, to be evaluated to a value provided by Kubernetes at Pod creation.
The default time zone for timestamps without an explicit zone.
Determines if requests to the kube-apiserver can be served by a cache.
Configuration for the logstash source.
If a socket address is used, it must include a port.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
The maximum number of TCP connections that are allowed at any given time.
The namespace to use for logs. This overrides the global setting.
The size of the receive buffer used for each connection.
Configuration for the mongodb_metrics source.
Each endpoint must be in the Connection String URI Format.
If set to an empty string, no namespace is added to the metrics.
By default, mongodb is used.
Configuration for the mqtt source.
Configuration for the nats source.
The URL takes the form of nats://server:port.
If the port is not specified it defaults to 4222.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Configuration for NATS JetStream.
The namespace to use for logs. This overrides the global setting.
The NATS queue group to join.
An optional path that deserializes an empty string to None.
This value determines how many messages the NATS subscriber buffers before incoming messages are dropped.
See the async_nats documentation for more information.
Configuration for the nginx_metrics source.
Each endpoint must be a valid HTTP/HTTPS URI pointing to an NGINX instance that has the
ngx_http_stub_status_module module enabled.
If set to an empty string, no namespace is added to the metrics.
By default, nginx is used.
Configuration for the okta source.
The Okta subdomain to scrape
API token for authentication
The namespace to use for logs. This overrides the global setting.
A span of time, in fractional seconds.
The time to look back for logs. This is used to determine the start time of the first request (that is, the earliest log to fetch)
TLS configuration.
Configuration for the opentelemetry source.
Configuration for the opentelemetry gRPC server.
2 nested properties
An internet socket address, either IPv4 or IPv6.
Configuration for the opentelemetry HTTP server.
4 nested properties
An internet socket address, either IPv4 or IPv6.
Accepts the wildcard (*) character for headers matching a specified pattern.
Specifying "*" results in all headers included in the log event.
These headers are not included in the JSON payload if a field with a conflicting name exists.
[]Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
The namespace to use for logs. This overrides the global setting.
One major caveat here is that the incoming metrics will be parsed as logs but they will preserve the OTLP format. This means that components that work on metrics, will not be compatible with this output. However, these events can be forwarded directly to a downstream OTEL collector.
Configuration for the postgresql_metrics source.
Each endpoint must be in the Connection URI format.
Specifying "" includes metrics where datname is NULL.
This can be used in conjunction with include_databases.
If not set, metrics are collected from all databases. Specifying "" includes metrics where datname is
NULL.
This can be used in conjunction with exclude_databases.
Overrides the default namespace for the metrics emitted by the source.
Configuration of TLS when connecting to PostgreSQL.
Configuration for the prometheus_pushgateway source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Only applies to counters and histograms as gauges and summaries can't be meaningfully aggregated.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
Configuration for the prometheus_remote_write source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
Defines the behavior for handling conflicting metric metadata.
The URL path on which metric POST requests are accepted.
When enabled, any metric sample with a NaN value will be filtered out during parsing, preventing downstream processing of invalid metrics.
Configuration for the prometheus_scrape source.
Endpoints to scrape metrics from.
The tag value is the endpoint of the scraped instance.
If true, the new tag is not added if the scraped metric has the tag already. If false, the conflicting tag
is renamed by prepending exported_ to the original name.
This matches Prometheus’ honor_labels configuration.
The tag value is the host and port of the scraped instance.
One or more values for the same parameter key can be provided. The parameters provided in this option are
appended to any parameters manually provided in the endpoints option. This option is especially useful when
scraping the /federate endpoint.
{}A span of time, in fractional seconds.
Configuration for the pulsar source.
The endpoint to which the Pulsar client should connect to.
The Pulsar topic names to read events from.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
Authentication configuration.
Max count of messages in a batch.
The Pulsar consumer name.
Dead Letter Queue policy configuration.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
The namespace to use for logs. This overrides the global setting.
The broker follows descending priorities. For example, 0=max-priority, 1, 2,...
In Shared subscription type, the broker first dispatches messages to the max priority level consumers if they have permits. Otherwise, the broker considers next priority level consumers.
The Pulsar subscription name.
TLS options configuration for the Pulsar client.
Configuration for the redis source.
The Redis key to read messages from.
The URL must take the form of protocol://server:port/db where the protocol can either be redis or rediss for connections secured using TLS.
The Redis data type (list or channel) to use.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
Options for the Redis list data type.
The namespace to use for logs. This overrides the global setting.
The value is the Redis key that the event was read from.
By default, this is not set and the field is not automatically added.
Configuration for the socket source.
Unix domain socket configuration for the socket source.
A file path.
Configures how events are decoded from raw bytes. Note some decoders can also determine the event output type (log, metric, trace).
An optional path that deserializes an empty string to None.
The namespace to use for logs. This overrides the global setting.
Note: The file mode value can be specified in any numeric format supported by your configuration language, but it is most intuitive to use an octal number.
Configuration for the splunk_hec source.
Acknowledgement configuration for the splunk_hec source.
{
"enabled": null,
"max_pending_acks": 10000000,
"max_number_of_ack_channels": 1000000,
"max_pending_acks_per_channel": 1000000,
"ack_idle_cleanup": false,
"max_idle_time": 300
}6 nested properties
A channel is idling if it is not used for sending data or querying acknowledgement statuses.
Enables end-to-end acknowledgements.
Channels can potentially idle for longer than this setting but clients should not rely on such behavior.
Minimum of 1.
Minimum of 1.
Equivalent to the max_number_of_acked_requests_pending_query Splunk HEC setting.
Minimum of 1.
Equivalent to the max_number_of_acked_requests_pending_query_per_ack_channel Splunk HEC setting.
Minimum of 1.
An internet socket address, either IPv4 or IPv6.
Configuration of HTTP server keepalive parameters.
2 nested properties
A value of 0.1 means that the actual duration will be between 90% and 110% of the specified maximum duration.
Only applies to HTTP/0.9, HTTP/1.0, and HTTP/1.1 requests.
A random jitter configured by max_connection_age_jitter_factor is added
to the specified duration to spread out connection storms.
The namespace to use for logs. This overrides the global settings.
If set to true, when incoming requests contain a Splunk HEC token, the token used is kept in the
event metadata and preferentially used if the event is sent to a Splunk HEC sink.
If supplied, incoming requests must supply this token in the Authorization header, just as a client would if
it was communicating with the Splunk HEC endpoint directly.
If not supplied, the Authorization header is ignored and requests are not authenticated.
If supplied, incoming requests must supply one of these tokens in the Authorization header, just as a client
would if it was communicating with the Splunk HEC endpoint directly.
If not supplied, the Authorization header is ignored and requests are not authenticated.
Configuration for the static_metrics source.
A span of time, in fractional seconds.
Tag configuration for the internal_metrics source.
[]Overrides the default namespace for the metrics emitted by the source.
Specifies the target unit for converting incoming StatsD timing values. When set to "seconds" (the default), timing values in milliseconds (ms) are converted to seconds (s). When set to "milliseconds", the original timing values are preserved.
Configuration for the statsd source.
Configuration for the syslog source.
Content encoding.
HTTP method.
Configuration of multi-line aggregation.
This setting must be configured in conjunction with mode.
Mode of operation of the line aggregator.
Regular expression pattern that is used to match the start of a new message.
If a socket address is used, it must include a port.
Configuration for the vector source.
An internet socket address, either IPv4 or IPv6.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
1 nested properties
Whether or not end-to-end acknowledgements are enabled for this source.
The namespace to use for logs. This overrides the global setting.
Version of the configuration.
Configuration for the websocket source.
In many cases, components can be configured so that part of the component's functionality can be customized on a per-event basis. For example, you have a sink that writes events to a file and you want to specify which file an event should go to by using an event field as part of the input to the filename used.
By using Template, users can specify either fixed strings or templated strings. Templated strings use a common syntax to
refer to fields in an event that is used as the input data when rendering the template. An example of a fixed string
is my-file.log. An example of a template string is my-file-{{key}}.log, where {{key}}
is the key's value when the template is rendered into a string.
The source of a uint template. May be a constant numeric value or a template string.
The source of a uint template. May be a constant numeric value or a template string.
Configurable transforms in Vector.
Configuration for the aggregate transform.
During this time frame, metrics (beta) with the same series data (name, namespace, tags, and so on) are aggregated.
Some of the functions may only function on incremental and some only on absolute metrics.
Configuration for the aws_ec2_metadata transform.
Overrides the default EC2 metadata endpoint.
A list of metadata fields to include in each transformed event.
[
"ami-id",
"availability-zone",
"instance-id",
"instance-type",
"local-hostname",
"local-ipv4",
"public-hostname",
"public-ipv4",
"region",
"subnet-id",
"vpc-id",
"role-name"
]Sets a prefix for all event fields added by the transform.
Configure to proxy traffic through an HTTP(S) proxy when making external requests.
Similar to common proxy configuration convention, you can set different proxies to use based on the type of traffic being proxied. You can also set specific hosts that should not be proxied.
4 nested properties
Enables proxying support.
Must be a valid URI string.
Must be a valid URI string.
Multiple patterns are allowed:
| Pattern | Example match |
|---|---|
| Domain names | example.com matches requests to example.com |
| Wildcard domains | .example.com matches requests to example.com and its subdomains |
| IP addresses | 127.0.0.1 matches requests to 127.0.0.1 |
| CIDR blocks | 192.168.0.0/16 matches requests to any IP addresses in this range |
| Splat | * matches all hosts |
[]Requires the transform to be able to successfully query the EC2 metadata before starting to process the data.
A list of instance tags to include in each transformed event.
[]Configuration for the dedupe transform.
Caching configuration for deduplication.
{
"num_events": 5000
}1 nested properties
Number of events to cache and use for comparing incoming events to previously seen events.
When no field matching configuration is specified, events are matched using the timestamp,
host, and message fields from an event. The specific field names used are those set in
the global log schema configuration.
Configuration for time based cache.
Configuration for the route transform.
An array of named routes. The route names are expected to be unique. Routes are evaluated in order from first to last, and only the first matching route receives each event (first-match-wins).
Configuration for the filter transform.
Many methods exist for matching events, such as using a VRL expression, a Datadog Search query string, or hard-coded matchers like "must be a metric" or "fields A, B, and C must match these constraints".
As VRL is the most common way to apply conditions to events, this type provides a shortcut to define VRL expressions directly in the configuration by passing the VRL expression as a string:
condition = '.message == "hooray"'
When other condition types are required, they can be specified with an enum-style notation:
condition.type = 'datadog_search'
condition.source = 'NOT "foo"'
Configuration for the incremental_to_absolute transform.
By default, incremental metrics are evicted after 5 minutes of not being updated. The next incremental value will be reset.
{
"max_bytes": null,
"max_events": null,
"time_to_live": null
}3 nested properties
The maximum size in bytes of the events in the metrics normalizer cache, excluding cache overhead.
The maximum number of events of the metrics normalizer cache
The maximum age of a metric not being updated before it is evicted from the metrics normalizer cache.
Configuration for the log_to_metric transform.
Example:
{
"counter": {
"value": 10.0
},
"kind": "incremental",
"name": "test.transform.counter",
"tags": {
"env": "test_env",
"host": "localhost"
}
}
This is a JSON representation of a counter with the following properties:
counter: An object with a single propertyvaluerepresenting the counter value, in this case,10.0).kind: A string indicating the kind of counter, in this case, "incremental".name: A string representing the name of the counter, here set to "test.transform.counter".tags: An object containing additional tags such as "env" and "host".
Objects that can be processed include counter, histogram, gauge, set and summary.
A list of metrics to generate.
Configuration for the lua transform.
Configuration for the metric_to_log transform.
If present, the value of the tag is set on the generated log event in the host field,
where the field key uses the global host_key option.
The namespace to use for logs. This overrides the global setting.
The user configuration to choose the metric tag strategy.
This overrides the global timezone option. The time zone name may be
any name in the TZ database or local to indicate system local time.
Configuration for the reduce transform.
If supplied, every time this interval elapses for a given grouping, the reduced value for that grouping is flushed. Checked every flush_period_ms.
If this condition resolves to true for an event, the current transaction is immediately
flushed with this event.
Each group with matching values for the specified keys is reduced independently, allowing
you to keep independent event streams separate. Note that each field specified, will be reduced
with the default merge strategy based on its type unless a merge strategy is explicitly defined
in merge_strategies.
This field is optional and when not specified, all events are reduced in a single group.
For example, if group_by = ["host", "region"], then all incoming events that have the same
host and region are grouped together before being reduced.
[]The maximum number of events to group together.
For each field specified, the given strategy is used for combining events rather than the default behavior.
The default behavior is as follows:
- The first value of a string field is kept and subsequent values are discarded.
- For timestamp fields the first is kept and a new field
[field-name]_endis added with the last received timestamp value. - Numeric values are summed.
- For nested paths, the field value is retrieved and then reduced using the default strategies mentioned above (unless explicitly specified otherwise).
{}If this condition resolves to true for an event, the previous transaction is flushed
(without this event) and a new transaction is started.
Configuration for the remap transform.
If a VRL program is manually aborted (using abort) when
processing an event, this option controls whether the original, unmodified event is sent
downstream without any modifications or if it is dropped.
Additionally, dropped events can potentially be diverted to a specially-named output for
further logging and analysis by setting reroute_dropped.
Normally, if a VRL program encounters an error when processing an event, the original,
unmodified event is sent downstream. In some cases, you may not want to send the event
any further, such as if certain transformation or enrichment is strictly required. Setting
drop_on_error to true allows you to ensure these events do not get processed any
further.
Additionally, dropped events can potentially be diverted to a specially named output for
further logging and analysis by setting reroute_dropped.
If a relative path is provided, its root is the current working directory.
Required if source is missing.
If a relative path is provided, its root is the current working directory.
Required if source or file are missing.
The user configuration to choose the metric tag strategy.
When using drop_on_error or drop_on_abort, events that are "dropped" are processed no
further. In some cases, it may be desirable to keep the events around for further analysis,
debugging, or retrying.
In these cases, reroute_dropped can be set to true which forwards the original event
to a specially-named output, dropped. The original event is annotated with additional
fields describing why the event was dropped.
The runtime to use for executing VRL code.
Required if file is missing.
This overrides the global timezone option. The time zone name may be
any name in the TZ database, or local to indicate system local time.
Configuration for the route transform.
Normally, if an event doesn't match any defined route, it is sent to the <transform_name>._unmatched
output for further processing. In some cases, you may want to simply discard unmatched events and not
process them any further.
In these cases, reroute_unmatched can be set to false to disable the <transform_name>._unmatched
output and instead silently discard any unmatched events.
The following identifiers are reserved output names and thus cannot be used as route IDs:
_unmatched_default
Each route can then be referenced as an input by other components with the name
<transform_name>.<route_id>. If an event doesn’t match any route, and if reroute_unmatched
is set to true (the default), it is sent to the <transform_name>._unmatched output.
Otherwise, the unmatched event is instead silently discarded.
Configuration for the sample transform.
A logical condition used to exclude events from sampling.
If left unspecified, or if the event doesn't have group_by, then the event is not
sampled separately.
Each unique value for the key creates a bucket of related events to be sampled together
and the rate is applied to the buckets themselves to sample 1/N buckets. The overall rate
of sampling may differ from the configured one if values in the field are not uniformly
distributed. If left unspecified, or if the event doesn’t have key_field, then the
event is sampled independently.
This can be useful to, for example, ensure that all logs for a given transaction are
sampled together, but that overall 1/N transactions are sampled.
For example, rate = 1500 means 1 out of every 1500 events are forwarded and the rest are
dropped. This differs from ratio which allows more precise control over the number of events
retained and values greater than 1/2. It is an error to provide a value for both rate and ratio.
For example, ratio = .13 means that 13% out of all events on the stream are forwarded and
the rest are dropped. This differs from rate allowing the configuration of a higher
precision value and also the ability to retain values of greater than 50% of all events. It is
an error to provide a value for both rate and ratio.
An optional path that deserializes an empty string to None.
Bloom filter configuration in probabilistic mode.
The larger the cache size, the less likely it is to have a false positive, or a case where we allow a new value for tag even after we have reached the configured limits.
Configuration for the tag_cardinality_limit transform.
Configuration for the tag_cardinality_limit transform for a specific group of metrics.
Configuration of internal metrics for the TagCardinalityLimit transform.
This helps identify which metrics and tag keys are hitting cardinality limits, but can significantly
increase metric cardinality. Defaults to false because these tags have potentially unbounded cardinality.
Possible actions to take when an event arrives that would exceed the cardinality limit for one or more of its tags.
Controls the approach taken for tracking tag cardinality.
Configuration for the throttle transform.
Each unique key has its own threshold.
A span of time, in fractional seconds.
A logical condition used to exclude events from sampling.
Configuration of internal metrics for the Throttle transform.
{
"emit_events_discarded_per_key": false
}1 nested properties
If true, the counter will be incremented for each discarded event, including the key value
associated with the discarded event. If false, the counter will not be emitted. Instead, the
number of discarded events can be seen through the component_discarded_events_total internal
metric.
Note that this defaults to false because the key tag has potentially unbounded cardinality.
Only set this to true if you know that the number of unique keys is bounded.
If left unspecified, or if the event doesn't have key_field, then the event is not rate
limited separately.
This is a naive implementation that simply converts a TraceEvent to a LogEvent.
The conversion preserves all trace attributes (span IDs, trace IDs, etc.) as log fields without modification.
This will need to be updated when Vector's trace data model is finalized to properly handle trace-specific semantics and field mappings.
The namespace to use for logs. This overrides the global setting.
Configuration for the window transform.
Many methods exist for matching events, such as using a VRL expression, a Datadog Search query string, or hard-coded matchers like "must be a metric" or "fields A, B, and C must match these constraints".
As VRL is the most common way to apply conditions to events, this type provides a shortcut to define VRL expressions directly in the configuration by passing the VRL expression as a string:
condition = '.message == "hooray"'
When other condition types are required, they can be specified with an enum-style notation:
condition.type = 'datadog_search'
condition.source = 'NOT "foo"'
If the condition resolves to true for an event, the event is immediately forwarded without
buffering and without preserving the original order of events. Use with caution if the sink
cannot handle out of order events.
The maximum number of events to keep after the event matched by the flush_when condition.
The maximum number of events to keep before the event matched by the flush_when condition.
Event handling behavior when a buffer is full.
A specific type of buffer stage.
Enumeration to define exactly what terms the bounds of the buffer is expressed in: length, or
byte_size.
Component identifier.
Wildcards (*) are supported.
See configuration for more info.
Wrapper for sensitive strings containing credentials
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
When enabled for a sink, any source that supports end-to-end acknowledgements that is connected to that sink waits for events to be acknowledged by all connected sinks before acknowledging them at the source.
Enabling or disabling acknowledgements at the sink level takes precedence over any global
acknowledgements configuration.
This setting is deprecated in favor of enabling acknowledgements at the global or sink level.
Enabling or disabling acknowledgements at the source level has no effect on acknowledgement behavior.
See End-to-end Acknowledgements for more information on how event acknowledgement is handled.
Whether or not end-to-end acknowledgements are enabled for this source.
Configuration for metric labels matcher.
Configure to proxy traffic through an HTTP(S) proxy when making external requests.
Similar to common proxy configuration convention, you can set different proxies to use based on the type of traffic being proxied. You can also set specific hosts that should not be proxied.
Enables proxying support.
Must be a valid URI string.
Must be a valid URI string.
Multiple patterns are allowed:
| Pattern | Example match |
|---|---|
| Domain names | example.com matches requests to example.com |
| Wildcard domains | .example.com matches requests to example.com and its subdomains |
| IP addresses | 127.0.0.1 matches requests to 127.0.0.1 |
| CIDR blocks | 192.168.0.0/16 matches requests to any IP addresses in this range |
| Splat | * matches all hosts |
[]A metric.
Metrics can be either absolute or incremental. Absolute metrics represent a sort of "last write wins" scenario, where the latest absolute value seen is meant to be the actual metric value. In contrast, and perhaps intuitively, incremental metrics are meant to be additive, such that we don't know what total value of the metric is, but we know that we'll be adding or subtracting the given value from it.
Generally speaking, most metrics storage systems deal with incremental updates. A notable exception is Prometheus, which deals with, and expects, absolute values from clients.
Metric data.
Metric time.
Intervals represent the time window over which this metric applies, and is generally only used for tracking rates (change over time) on counters.
Metrics may sometimes have no timestamp, or have no meaningful value if the metric is an aggregation or transformed heavily enough from its original form such that the original timestamp would not represent a meaningful value.
Metric name.
This would typically be a name for the metric itself, unrelated to where the metric
originates from. For example, if the metric represented the amount of used system memory, it
may be called memory.used.
Namespace represents a grouping for a metric where the name itself may otherwise be too
generic. For example, while the name of a metric may be memory.used for the amount of used
system memory, the namespace could differentiate that by being system for the total amount
of used memory across the system, or vector for the amount of used system memory specific
to Vector, and so on.
Metrics series.
Tags for a metric series.
A single tag value, either a bare tag or a value.
Tag values for a metric series. This may be empty, a single value, or a set of values. This is
used to provide the storage for TagValueSet.
Histogram buckets represent the count of observations where the value of the observations does
not exceed the specified upper_limit.
The number of values tracked in this bucket.
The upper limit of values in the bucket.
A generalized metrics sketch.
Metric value. Container for the actual value of a metric.
Quantiles themselves are "cut points dividing the range of a probability distribution into continuous intervals with equal probabilities". [1].
We use quantiles to measure the value along these probability distributions for representing client-side aggregations of distributions, which represent a collection of observations over a specific time window.
In general, we typically use the term "quantile" to represent the concept of percentiles, which deal with whole integers -- 0, 1, 2, .., 99, 100 -- even though quantiles are floating-point numbers and can represent higher-precision cut points, such as 0.9999, or the 99.99th percentile.
This value must be between 0.0 and 1.0, inclusive.
The estimated value of the given quantile within the probability distribution.
A single observation.
The rate at which the value was observed.
The value of the observation.
Type of statistics to generate for a distribution.
List of allowed origin IP networks. IP addresses must be in CIDR notation.
This implementation is subtly different from the open-source implementations of DDSketch, as
Datadog made some slight tweaks to configuration values and in-memory layout to optimize it for
insertion performance within the agent.
We've mimicked the agent version of DDSketch here in order to support a future where we can
take sketches shipped by the agent, handle them internally, merge them, and so on, without any
loss of accuracy, eventually forwarding them to Datadog ourselves.
As such, this implementation is constrained in the same ways: the configuration parameters cannot be changed, the collapsing strategy is fixed, and we support a limited number of methods for inserting into the sketch.
Importantly, we have a special function, again taken from the agent version, to allow us to interpolate histograms, specifically our own aggregated histograms, into a sketch so that we can emit useful default quantiles, rather than having to ship the buckets -- upper bound and count -- to a downstream system that might have no native way to do the same thing, basically providing no value as they have no way to render useful data from them.
The average value of all observations within the sketch.
While internally we depend on a vector of bins, the serialized form used in Protocol Buffers splits the bins -- which contain their own key and bin count -- into two separate vectors, one for the keys and one for the bin counts.
This type provides the glue to go from our internal representation to the Protocol Buffers-specific representation.
2 nested properties
The bin keys.
The bin counts.
The number of observations within the sketch.
The maximum value of all observations within the sketch.
The minimum value of all observations within the sketch.
The sum of all observations within the sketch.
While internally we depend on a vector of bins, the serialized form used in Protocol Buffers splits the bins -- which contain their own key and bin count -- into two separate vectors, one for the keys and one for the bin counts.
This type provides the glue to go from our internal representation to the Protocol Buffers-specific representation.
The bin keys.
The bin counts.
TCP keepalive settings for socket-based components.
The time to wait before starting to send TCP keepalive probes on an idle connection.
TLS configuration.
Declare the supported ALPN protocols, which are used during negotiation with a peer. They are prioritized in the order that they are defined.
The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format.
The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format.
If this is set and is not a PKCS#12 archive, key_file must also be set.
The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format.
This has no effect unless key_file is set.
Only relevant for outgoing connections.
If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on, until the verification process reaches a root certificate.
Do NOT set this to false unless you understand the risks of not verifying the validity of certificates.
If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension.
Only relevant for outgoing connections.
Do NOT set this to false unless you understand the risks of not verifying the remote hostname.
Configures the TLS options for incoming/outgoing connections.
TlsEnableableConfig for sources, adding metadata from the client certificate.
A wrapper around OwnedTargetPath that allows it to be used in Vector config
with prefix default to PathPrefix::Event
A wrapper around OwnedValuePath that allows it to be used in Vector config.
This requires a valid path to be used. If you want to allow optional paths,
use [optional_path::OptionalValuePath].
An optional path that deserializes an empty string to None.
An optional path that deserializes an empty string to None.
This can refer to any valid timezone as defined in the TZ database, or "local" which refers to the system local timezone. It will default to the globally configured timezone.