The transaction command in Splunk isn’t just about grouping logs; it’s about reconstructing a complete user journey from disparate events, often revealing critical context that individual logs miss entirely.

Let’s see it in action. Imagine you have web server logs and application logs, and you want to see complete user requests from start to finish.

Here’s a snippet of raw data:

# Web Server Access Log (Apache format)
192.168.1.10 - - [10/Oct/2023:10:00:01 +0000] "GET /login HTTP/1.1" 200 1234 "-" "Mozilla/5.0"
192.168.1.10 - - [10/Oct/2023:10:00:05 +0000] "POST /authenticate HTTP/1.1" 200 567 "-" "Mozilla/5.0"
192.168.1.10 - - [10/Oct/2023:10:00:10 +0000] "GET /dashboard HTTP/1.1" 200 890 "-" "Mozilla/5.0"
192.168.1.10 - - [10/Oct/2023:10:00:15 +0000] "GET /logout HTTP/1.1" 200 123 "-" "Mozilla/5.0"

# Application Log
[2023-10-10 10:00:05,123] INFO [auth-service] User 'user123' authenticated successfully.
[2023-10-10 10:00:06,456] DEBUG [auth-service] Session ID generated: abcdef123456
[2023-10-10 10:00:11,789] INFO [dashboard-service] User 'user123' accessed dashboard.

To correlate these, we need to identify common fields and define what constitutes a "transaction." A common scenario is tracking a user’s session. We can use the remote_addr (IP address) to group events and the uri_path to define the start and end of a request sequence.

The transaction command works by taking a set of events and grouping them into "transactions" based on specified criteria. You define a transaction by identifying a startswith condition and an endswith condition, and optionally, a maxspan to limit the duration of a transaction and a maxpause to define the maximum time allowed between events within a single transaction. Events that fall between the start and end, or are within the maxspan and maxpause, are considered part of the same transaction.

Here’s how you’d use it to analyze web requests, assuming you have both web server and application logs indexed, and you’ve extracted fields like remote_addr, uri_path, status, and session_id:

index=web OR index=app
| transaction remote_addr startswith="GET /login" endswith="GET /logout" maxspan=5m maxpause=1m
| table _time, remote_addr, uri_path, status, session_id, duration, eventcount

In this search:

  • index=web OR index=app: We’re looking at events from both our web server and application logs.
  • transaction remote_addr startswith="GET /login" endswith="GET /logout": This is the core. We’re grouping events by remote_addr. A transaction starts when an event contains "GET /login" and ends when an event contains "GET /logout". All events in between, from the same remote_addr, will be bundled.
  • maxspan=5m: If a user’s session spans more than 5 minutes between the very first event (/login) and the very last (/logout), it will be considered a new transaction or not grouped.
  • maxpause=1m: If there’s more than a 1-minute gap between consecutive events within a potential transaction, Splunk will consider it a new transaction. This is crucial for identifying periods of inactivity.
  • table _time, remote_addr, uri_path, status, session_id, duration, eventcount: This displays the results. _time is the start time of the transaction, duration is how long the transaction took, and eventcount is the number of events included. We’re also pulling in uri_path, status, and session_id to see the details of the events within the transaction.

The power of transaction lies in its ability to stitch together related events into a single, actionable unit. Instead of seeing a flurry of individual web requests and application logs, you see a single entry representing a user’s entire login-to-logout journey, including the time it took, the number of steps, and any associated session information. This is invaluable for understanding user behavior, debugging complex workflows, and identifying performance bottlenecks.

One of the most surprising aspects of the transaction command is how it handles fields that change within a transaction. By default, Splunk will only retain the first value of a field encountered in the transaction. If you need to see a range or all values of a field that changes, you often need to use aggregation commands like values() or list() in conjunction with transaction, or pre-process your data to create a consistent field that spans the transaction (e.g., a session ID derived from the first event that is then applied to all subsequent events in that session using subsearches or join).

The next logical step after correlating sessions is to analyze the performance of these transactions, often leading you to the stats or timechart commands to aggregate transaction durations and identify outliers.

Want structured learning?

Take the full Splunk course →