Splunk

Splunk Integration Guide

Splunk collects and indexes data from just about any source imaginable – network traffic, Web servers, custom applications, application servers, hypervisors, GPS systems, stock market feeds, social media, and preexisting structured databases. Integrate AlertOps’ alert management platform with Splunk to receive and respond to critical alerts through email, SMS, push notification, and phone alerts. AlertOps ensures that alerts received from Splunk always reach the correct, available team member by utilizing escalation policies and on-call schedules.

AlertOps Configuration

1. From the main menu, click on Integrations and then select Inbound Integrations from the sub menu.
2. Select API tab
3. Select the 'ADD API INTEGRATION' button, you should now be on the API Integration Detail page.
4. Select Splunk from the pre-built Integration Templates.
5. Enter a name for the integration. Select an escalation rule to determine the integration's escalation policy.
6. Enter the names of recipient group(s) and recipient user(s).
7. Click 'SAVE.' Then, click 'COPY URL' to copy the URL endpoint to be used in the Splunk configuration.

Splunk Configuration

• Log into Splunk (Enterprise) and go to the Search & Reporting app (or equivalent).
• Create a search query that detects the condition you care about (for example, errors, threshold breaches).
• Click Save As → Alert.

• Define the trigger condition (when to fire, e.g., “if count > 0”)
• Under Trigger Actions, add Webhook (or “Send to Webhook”) as an action. 

• Paste in the Webhook URL you copied from AlertOps. Save ! 
• (Optional) If it is Splunk Observability Cloud: You can choose 'Data Management' > 'Available Integrations' > find 'Webhook'

Add the configurations as shown, you can leave the rest of the sections default.

• Review and Save
• Ensure you associate the webhook as an Alert Recipient to the Alert Rule, that you may have defined. 

For Splunk Observability Cloud, the Payload sent out would be different, this should be mapped under 'Rules for Opening and Closing Alert' under Advanced Settings of the Splunk Integration in AlertOps (Source Id, Source URL, Subject, Source Status, Long Text and Short Text)

Alert Triggering Information

AlertOps will automatically create an incident when a new alert is received from Splunk.

Testing and Troubleshooting

Click here to read about Web API Testing and Troubleshooting.