Automatic namespacing for configuration files
New automatic namespacing functionality to better organize your Vector configuration files
We’ve released automatic namespacing for configuration files, which simplifies namespace configuration for your pipelines.
As Vector continues to evolve, releasing more configuration-heavy functionality such as an aggregator role and pipelines features, there’s often a proliferation in the amount of configuration necessary to run Vector. The ability to organize Vector across multiple files was also lacking in any concrete recommendations for collaboration and navigation; Vector users may have dozens of Vector configuration files, from multiple source files to countless sink files, in a single directory.
Let’s look at how we can refactor an existing set of Vector configuration files.
Assuming Vector is loaded using the configuration via
/etc/vector/ │ file001.toml │ file002.toml │ file003.toml │ ... │ file022.toml │ file023.toml
We can update this using Vector’s new automatic namespacing to organize the configuration into separate files based on Vector’s configuration directory structure. This makes it easy for users like you to split up your configuration files and collaborate with others on their team.
--config-dir to provide Vector’s configuration, it will look for
subdirectories with the component types (e.g.
automatically infer that the files in these directories refer to that type of
component. Further, Vector will use filenames of the configuration files as
their component ID. The end result is that you have to specify less
configuration as Vector can infer the component type and id from the directory
Using the above example, now with automatic namespacing, you can update this to look like:
/etc/vector/ └───sources/ │ │ file001.toml │ │ ... │ │ file005.toml │ └───transforms/ │ │ file006.toml │ │ ... │ │ file016.toml │ └───sinks/ │ file017.toml │ file022.toml
The configuration files become simplified from:
# /etc/vector/file017.toml [sinks.foo] type = "anything"
# /etc/vector/sinks/file017.toml type = "anything"