Vector Log to Metric: Convert Events to Prometheus Metrics (2026)

Vector, the observability data pipeline, can transform your logs into Prometheus metrics, giving you the power to monitor events that aren’t explicitly exposed as metrics.

Let’s see it in action. Imagine you have a web server log file like this:

192.168.1.10 - - [10/Oct/2023:14:55:36 +0000] "GET /api/users HTTP/1.1" 200 1234 "-" "curl/7.68.0"
192.168.1.11 - - [10/Oct/2023:14:55:37 +0000] "POST /api/orders HTTP/1.1" 201 56 "-" "python-requests/2.25.1"
192.168.1.10 - - [10/Oct/2023:14:55:38 +0000] "GET /api/users HTTP/1.1" 404 0 "-" "curl/7.68.0"
192.168.1.12 - - [10/Oct/2023:14:55:39 +0000] "GET /api/users HTTP/1.1" 200 1234 "-" "curl/7.68.0"

We want to count the number of successful requests (2xx and 3xx status codes) and the number of failed requests (4xx and 5xx status codes) per endpoint.

Here’s a Vector configuration that accomplishes this:

[sources.apache_logs]
type = "file"
include = ["/var/log/apache2/access.log"] # Adjust path as needed
content_type = "apache_common" # Or "apache_combined" if your logs include response size and referrer

[transforms.parse_log]
type = "remap"
inputs = ["apache_logs"]
source = '''
. = parse_apache_log!(.message)
'''

[transforms.filter_success]
type = "filter"
inputs = ["parse_log"]
# Keep events where the status code is >= 200 and < 400
condition = ".status >= 200 && .status < 400"

[transforms.count_success]
type = "aggregation"
inputs = ["filter_success"]
# Group by the request path and the status code
group_by = [".request.path", ".status"]
# Count the number of occurrences for each group
metrics = { success_requests_total = { type = "counter", value = 1.0 } }

[transforms.filter_failure]
type = "filter"
inputs = ["parse_log"]
# Keep events where the status code is >= 400
condition = ".status >= 400"

[transforms.count_failure]
type = "aggregation"
inputs = ["filter_failure"]
# Group by the request path and the status code
group_by = [".request.path", ".status"]
# Count the number of occurrences for each group
metrics = { failed_requests_total = { type = "counter", value = 1.0 } }

[sinks.prometheus_output]
type = "prometheus_exporter"
inputs = ["count_success", "count_failure"]
# Expose metrics on port 9090
listen_address = "0.0.0.0:9090"
# Namespace for your metrics
namespace = "my_web_app"

In this configuration:

• sources.apache_logs: This file source reads from your Apache access log. content_type = "apache_common" tells Vector how to interpret the log lines.
• transforms.parse_log: The remap transform uses the parse_apache_log! built-in function to parse each log line into structured data. This makes fields like .status, .request.path, etc., available.
• transforms.filter_success and transforms.filter_failure: These filter transforms act as routers. One keeps lines with successful status codes (2xx, 3xx), and the other keeps lines with failed status codes (4xx, 5xx).
• transforms.count_success and transforms.count_failure: These aggregation transforms are the core of the metric generation.
  • group_by = [".request.path", ".status"] tells Vector to create separate metrics for each unique combination of request path and status code.
  • metrics = { success_requests_total = { type = "counter", value = 1.0 } } defines a Prometheus counter metric named success_requests_total. For every log line that passes through this transform, its value is incremented by 1.0. The same logic applies to failed_requests_total.
• sinks.prometheus_output: This prometheus_exporter sink exposes the generated metrics over HTTP on port 9090. Prometheus can then scrape this endpoint. The namespace = "my_web_app" prefixes all generated metrics, preventing naming collisions.

When Prometheus scrapes http://your-vector-host:9090/metrics, you’ll see output like this:

# HELP my_web_app_success_requests_total Total successful requests.
# TYPE my_web_app_success_requests_total counter
my_web_app_success_requests_total{request_path="/api/users",status="200"} 2
my_web_app_success_requests_total{request_path="/api/orders",status="201"} 1
# HELP my_web_app_failed_requests_total Total failed requests.
# TYPE my_web_app_failed_requests_total counter
my_web_app_failed_requests_total{request_path="/api/users",status="404"} 1

The true power here is that you’re not limited to parsing known log formats. If you have custom application logs with events like "user_login_failed" or "database_connection_error", you can parse these messages, filter for the specific event, and increment a counter.

The most surprising thing is that the aggregation transform’s value field doesn’t have to be 1.0. You can actually sum up values from your logs. For example, if your logs contained request durations, you could use metrics = { request_duration_seconds_sum = { type = "counter", value = ".duration_ms" } } to sum up the durations (after converting ms to seconds, of course). This allows you to create metrics like http_request_duration_seconds_sum directly from unstructured log data.

With this foundation, you can start to build a comprehensive view of your application’s behavior, even for events that aren’t explicitly instrumented.

Next, you’ll likely want to explore how to add more complex labels to your metrics, such as the user agent or IP address, and how to sample your logs to avoid overwhelming your Vector instance.