dnstap
Collect DNS logs from a dnstap-compatible server
Configuration
Example configurations
{
"sources": {
"my_source_id": {
"type": "dnstap"
}
}
}
[sources.my_source_id]
type = "dnstap"
sources:
my_source_id:
type: dnstap
{
"sources": {
"my_source_id": {
"type": "dnstap",
"max_frame_length": 102400,
"socket_file_mode": 511
}
}
}
[sources.my_source_id]
type = "dnstap"
max_frame_length = 102_400
socket_file_mode = 511
sources:
my_source_id:
type: dnstap
max_frame_length: 102400
socket_file_mode: 511
host_key
optional string literalOverrides the name of the log field used to add the source path to each event.
The value is the socket path itself.
By default, the global log_schema.host_key
option is used.
max_frame_handling_tasks
optional uintmax_frame_length
optional uintMaximum DNSTAP frame length that the source accepts.
If any frame is longer than this, it is discarded.
102400
(bytes)multithreaded
optional boolraw_data_only
optional boolWhether or not to skip parsing or decoding of DNSTAP frames.
If set to true
, frames are not parsed or decoded. The raw frame data is set as a field on the event
(called rawData
) and encoded as a base64 string.
socket_file_mode
optional uintUnix file mode bits to be applied to the unix socket file as its designated file permissions.
Note: The file mode value can be specified in any numeric format supported by your configuration language, but it is most intuitive to use an octal number.
socket_path
required string literalAbsolute path to the socket file to read DNSTAP data from.
The DNS server must be configured to send its DNSTAP data to this socket file. The socket file is created if it doesn’t already exist when the source first starts.
socket_receive_buffer_size
optional uintThe size, in bytes, of the receive buffer used for the socket.
This should not typically needed to be changed.
socket_send_buffer_size
optional uintThe size, in bytes, of the send buffer used for the socket.
This should not typically needed to be changed.
Outputs
<component_id>
Output Data
Logs
Warning
Event
1
Encountered error : Unexpected number of records in update section: 0
an arbitrary byte-string annotation
6
ChBqYW1lcy11YnVudHUtZGV2EgtCSU5EIDkuMTYuNXKdAQgCEAEYASIEfwAAASoEfwAAATDRyAM4AFoNB2V4YW1wbGUDY29tAGCTvf76BW3evGImcmlihYQAAAEAAAABAAACaDIHZXhhbXBsZQNjb20AAAYAAcAPAAYAAQAADhAAPQtiZGRzLWRuc3RhcAAKcG9zdG1hc3RlcgJubwVlbWFpbAZwbGVhc2UAJADGPgAADhAAAAJYACeNAAAADhB4AQ==
192.0.2.18
fc00::200
60364
ns1.example.com
BIND 9.16.8
INET
INET6
UDP
TCP
192.0.2.8
fc00::100
52398
1614781642516276700
s
ms
us
ns
2021-04-09T15:08:32.767098Z
Telemetry
Metrics
linkcomponent_discarded_events_total
countercomponent_id
instead. The value is the same as component_id
.filter
transform, or false if due to an error.component_errors_total
countercomponent_id
instead. The value is the same as component_id
.component_received_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_events_count
histogramA histogram of the number of events passed in each internal batch in Vector’s internal topology.
Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.
component_id
instead. The value is the same as component_id
.component_received_events_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_events_total
countercomponent_id
instead. The value is the same as component_id
.parse_errors_total
countersource_lag_time_seconds
histogramcomponent_id
instead. The value is the same as component_id
.Examples
Dnstap events for a pair of regular DNS query and response.
Given this event...Send a query to an authoritative BIND DNS server locally with following command:
```bash
nslookup host.example.com localhost
```
sources:
my_source_id:
type: dnstap
max_frame_length: 102400
socket_file_mode: 508
socket_path: /run/bind/dnstap.sock
max_frame_handling_tasks: 10000
[sources.my_source_id]
type = "dnstap"
max_frame_length = 102_400
socket_file_mode = 508
socket_path = "/run/bind/dnstap.sock"
max_frame_handling_tasks = 10_000
{
"sources": {
"my_source_id": {
"type": "dnstap",
"max_frame_length": 102400,
"socket_file_mode": 508,
"socket_path": "/run/bind/dnstap.sock",
"max_frame_handling_tasks": 10000
}
}
}
[{"log":{"dataType":"Message","dataTypeId":1,"messageType":"ClientQuery","messageTypeId":5,"requestData":{"fullRcode":0,"header":{"aa":false,"ad":false,"anCount":0,"arCount":0,"cd":false,"id":49653,"nsCount":0,"opcode":0,"qdCount":1,"qr":0,"ra":false,"rcode":0,"rd":true,"tc":false},"question":[{"class":"IN","domainName":"host.example.com.","questionType":"A","questionTypeId":1}],"rcodeName":"NoError","time":1614781642516276700,"timePrecision":"ns"},"responseAddress":"127.0.0.1","responsePort":0,"serverId":"ns1.example.com","serverVersion":"BIND 9.16.8","socketFamily":"INET","socketProtocol":"UDP","sourceAddress":"127.0.0.1","sourcePort":52398,"time":1614781642516276700,"timePrecision":"ns"}},{"log":{"dataType":"Message","dataTypeId":1,"messageType":"ClientResponse","messageTypeId":6,"responseAddress":"127.0.0.1","responseData":{"answers":[{"class":"IN","domainName":"host.example.com.","rData":"192.0.2.100","recordType":"A","recordTypeId":1,"ttl":3600}],"authority":[{"class":"IN","domainName":"example.com.","rData":"ns1.example.com.","recordType":"NS","recordTypeId":2,"ttl":86400}],"fullRcode":0,"header":{"aa":true,"ad":false,"anCount":1,"arCount":0,"cd":false,"id":49653,"nsCount":1,"opcode":0,"qdCount":1,"qr":1,"ra":true,"rcode":0,"rd":true,"tc":false},"question":[{"class":"IN","domainName":"host.example.com.","questionType":"A","questionTypeId":1}],"rcodeName":"NoError","time":1614781642516276700,"timePrecision":"ns"},"responsePort":0,"serverId":"ns1.example.com","serverVersion":"BIND 9.16.8","socketFamily":"INET","socketProtocol":"UDP","sourceAddress":"127.0.0.1","sourceId":"421bce7d-b4e6-b705-6057-7039628a9847","sourcePort":52398,"time":1614781642516276700,"timePrecision":"ns"}}]
Dnstap events for a pair of DNS update request and response.
Given this event...Send a dynamic update to an authoritative BIND DNS server locally with following command:
```bash
nsupdate <<EOF
server localhost
update add h1.example.com 3600 a 192.0.2.110
send
EOF
```
sources:
my_source_id:
type: dnstap
socket_file_mode: 508
socket_path: /run/bind/dnstap.sock
socket_receive_buffer_size: 10485760
socket_send_buffer_size: 10485760
[sources.my_source_id]
type = "dnstap"
socket_file_mode = 508
socket_path = "/run/bind/dnstap.sock"
socket_receive_buffer_size = 10_485_760
socket_send_buffer_size = 10_485_760
{
"sources": {
"my_source_id": {
"type": "dnstap",
"socket_file_mode": 508,
"socket_path": "/run/bind/dnstap.sock",
"socket_receive_buffer_size": 10485760,
"socket_send_buffer_size": 10485760
}
}
}
[{"log":{"dataType":"Message","dataTypeId":1,"messageType":"UpdateQuery","messageTypeId":13,"requestData":{"fullRcode":0,"header":{"adCount":0,"id":47320,"opcode":5,"prCount":0,"qr":0,"rcode":0,"upCount":1,"zoCount":1},"rcodeName":"NoError","time":1599832089886768400,"timePrecision":"ns","update":[{"class":"IN","domainName":"h1.example.com.","rData":"192.0.2.110","recordType":"A","recordTypeId":1,"ttl":3600}],"zone":{"zClass":"IN","zName":"example.com.","zType":"SOA","zTypeId":6}},"responseAddress":"127.0.0.1","responsePort":0,"serverId":"ns1.example.com","serverVersion":"BIND 9.16.8","socketFamily":"INET","socketProtocol":"UDP","sourceAddress":"127.0.0.1","sourcePort":53141,"time":1599832089886768400,"timePrecision":"ns"}},{"log":{"dataType":"Message","dataTypeId":1,"messageType":"UpdateResponse","messageTypeId":14,"responseAddress":"127.0.0.1","responseData":{"fullRcode":0,"header":{"adCount":0,"id":47320,"opcode":5,"prCount":0,"qr":1,"rcode":0,"upCount":0,"zoCount":1},"rcodeName":"NoError","time":1599832089890768400,"timePrecision":"ns","zone":{"zClass":"IN","zName":"example.com.","zType":"SOA","zTypeId":6}},"responsePort":0,"serverId":"ns1.example.com","serverVersion":"BIND 9.16.8","socketFamily":"INET","socketProtocol":"UDP","sourceAddress":"127.0.0.1","sourcePort":53141,"time":1599832089890768400,"timePrecision":"ns"}}]
How it works
Manipulate UDS Buffer Size
The dnstap
source supports configuring the UDS buffer for both receiving and
sending, which may be helpful for handling DNS traffic spikes more smoothly in
high-usage scenarios in which performance is of paramount concern.
To configure the send/receive buffer size for the server UDS, set the
socket_receive_buffer_size
and
socket_send_buffer_size
parameters in the component’s
configuration. Here’s an example:
[sources.my_dnstap_source]
type = "dnstap"
socket_receive_buffer_size = 10_485_760
socket_send_buffer_size = 10_485_760
# Other configs
For the buffer size settings to take effect, you need to ensure that the system-wide
settings for send/receive buffer sizes (i.e. the values of
/proc/sys/net/core/rmem_max
and /proc/sys/net/core/wmem_max
on Linux) are
large enough.
Server Unix Domain Socket (UDS)
The dnstap
source receives dnstap data through a Unix Domain Socket (aka UDS). The
path of the UDS must be explicitly specified in the source’s configuration.
Upon startup, the dnstap
source creates a new server UDS at the specified path.
If the path of UDS is already in use, Vector automatically deletes it before
creating a new path.
The default permissions of the UDS are determined by the current umask
value.
To customize it to allow the local BIND server to send dnstap data to the UDS,
you can specify the desired UDS permissions (for example the file mode) explicitly
in the dnstap
source configuration. To set its permissions to 0774
, for example,
add the socket_file_mode
option:
[sources.my_dnstap_source]
type = "dnstap"
socket_file_mode: 0o774
# Other configs
Using a remote BIND server
dnstap
source can create server UDS paths only on the local
machine, you can also use it with remote BIND servers by forwarding the
server UDS from the machine Vector is running on to the remote BIND server
(for example via SSH) once Vector starts. Make sure that the Unix domain
sockets on both the local and remote machines have appropriate permissions
set.