Splunk’s Ingest Actions let you filter and route data before it ever hits your indexes, saving you money and making your data more relevant from the get-go.
Let’s see it in action. Imagine you’re getting a ton of verbose debug logs from a web application, but you only really care about the ERROR and WARN level messages. You can set up an Ingest Action to drop all the INFO and DEBUG messages.
Here’s a simplified example of what that might look like in a transforms.conf file, which is where you define these actions:
[webserver_filter]
REGEX = (ERROR|WARN)
DEST_KEY = _raw
# This is a simple example, more complex routing is possible
And in props.conf:
[my_web_app]
TRANSFORMS = webserver_filter
When data from your my_web_app source type hits Splunk, the webserver_filter transform is applied. If the REGEX (ERROR|WARN) doesn’t match the _raw event (the entire log line), the event is dropped. Only ERROR and WARN messages pass through to be indexed.
This isn’t just about dropping data; it’s about intelligent routing. You can use Ingest Actions to direct specific data to different indexes based on its content. For example, security-related events could be routed to a dedicated security index, while application logs go to an applications index.
Here’s how you might configure routing using props.conf and transforms.conf:
In transforms.conf:
[route_security_events]
REGEX = (failed login|unauthorized access|security alert)
DEST_KEY = _meta:index
# The value here tells Splunk where to send the event
TRANSFORM = set::index=security
[route_app_logs]
REGEX = .*
DEST_KEY = _meta:index
TRANSFORM = set::index=applications
And in props.conf:
[my_security_app]
TRANSFORMS = route_security_events, route_app_logs
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = \n
In this setup, my_security_app is a source type. When an event comes in, Splunk first checks route_security_events. If the REGEX matches, it sets the index to security. If it doesn’t match, it falls through to route_app_logs, which has a broad REGEX (.*) that matches everything and sets the index to applications. The order in TRANSFORMS matters.
The real power comes from combining filtering and routing. You can filter out noisy debug logs and route critical security events to a specific index, all before they consume valuable disk space and search time.
The DEST_KEY is crucial. _raw refers to the event’s raw data. _meta:index is a special key that allows you to dynamically set metadata like the target index. set::index=your_index_name is the syntax for assigning that index.
A common misconception is that Ingest Actions are purely for dropping data. While that’s a primary use case for cost savings, they are fundamentally a powerful data manipulation engine at the very edge of your Splunk ingestion pipeline. You can also use them to enrich events, add metadata, or even trigger external scripts (though that’s more advanced and less common for simple filtering/routing). The REGEX engine is robust, and you can build complex logic to precisely control what data enters your Splunk environment and where it lands.
The next step after mastering basic filtering and routing is often exploring how to use these actions to dynamically set other metadata fields, like sourcetype or host, based on event content.