Sample
Sample events from an event stream based on supplied criteria and at a configurable rate
Configuration
Example configurations
{
"transforms": {
"my_transform_id": {
"type": "sample",
"inputs": [
"my-source-or-transform-id"
]
}
}
}
[transforms.my_transform_id]
type = "sample"
inputs = [ "my-source-or-transform-id" ]
---
transforms:
my_transform_id:
type: sample
inputs:
- my-source-or-transform-id
{
"transforms": {
"my_transform_id": {
"type": "sample",
"inputs": [
"my-source-or-transform-id"
],
"key_field": "message"
}
}
}
[transforms.my_transform_id]
type = "sample"
inputs = [ "my-source-or-transform-id" ]
key_field = "message"
---
transforms:
my_transform_id:
type: sample
inputs:
- my-source-or-transform-id
key_field: message
exclude
optional conditionAvailable syntaxes
Syntax | Description | Example |
---|---|---|
vrl | A Vector Remap Language (VRL) Boolean expression. | .status_code != 200 && !includes(["info", "debug"], .severity) |
datadog_search | A Datadog Search query string. | *stack |
Shorthand for VRL
If you opt for the vrl
syntax for this condition, you can set the condition
as a string via the condition
parameter, without needing to specify both a source
and a type
. The
table below shows some examples:
Config format | Example |
---|---|
TOML | condition = ".status == 200" |
YAML | condition: .status == 200 |
JSON | "condition": ".status == 200" |
Condition config examples
Standard VRL
exclude = { type = "vrl", source = ".status == 500" }
exclude:
type: "vrl"
source: ".status == 500"
"exclude": {
"type": "vrl",
"source": ".status == 500"
}
inputs
required [string]A list of upstream source or transform IDs.
Wildcards (*
) are supported.
See configuration for more info.
key_field
optional string literalThe name of the log field whose value will be hashed to determine if the event should be passed.
Consistently samples the same events. Actual rate of sampling may differ from the configured
one if values in the field are not uniformly distributed. If left unspecified, or if the
event doesn’t have key_field
, events will be count rated.
rate
required uintThe rate at which events will be forwarded, expressed as 1/N
.
For example, rate = 10
means 1 out of every 10 events will be forwarded and the rest will
be dropped.
Outputs
<component_id>
Telemetry
Metrics
linkcomponent_received_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_events_count
histogramA histogram of the number of events passed in each internal batch in Vector’s internal topology.
Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.
component_id
instead. The value is the same as component_id
.component_received_events_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_events_total
countercomponent_id
instead. The value is the same as component_id
.events_discarded_total
counterevents_in_total
countercomponent_received_events_total
instead.component_id
instead. The value is the same as component_id
.events_out_total
countercomponent_sent_events_total
instead.component_id
instead. The value is the same as component_id
.processed_bytes_total
countercomponent_id
instead. The value is the same as component_id
.processed_events_total
countercomponent_received_events_total
and
component_sent_events_total
metrics.component_id
instead. The value is the same as component_id
.utilization
gaugecomponent_id
instead. The value is the same as component_id
.