The State of IT Automation: New Pressures Invite New Opportunities Read Report

The Definitive Guide for Responding to False Positives on Incident Response

The Definitive Guide for Responding to False Positives on Incident Response
September 20, 2017 • Resolve Staffer

Resolve Systems has seen firsthand the time-guzzling trap of false positives on enterprise-wide incident response for many companies. Does the following sound familiar?

Alerts pop up and Level 1 Security agents spend time [under pressure] trying to defuse a “threat” before passing right, to the subject-matter experts for [costlier] escalations. The SMEs apply their knowledge and the alert is identified as an IT issue and the buck continues to be passed. First to the IT L1, then, often again through the chain-gang of command.

employee trying to decipher false positive alters

False positives divert time and create stress when incident response for Cybersecurity and IT Ops work is already high pressure.

Wouldn’t it be ideal if incident response could be swiftly and succinctly parsed into false positives and legitimate issues? Here’s a definitive guide to common false positives and how Resolve Systems validates, integrates and remediates the daily deluge of alerts.

Beyond false positives and through alert fatigue

Here’s a look at one such success story. Zayo, a Fortune 500 global communications connectivity service provider, now automates manual health checks and quashes false positives within its global command center, thanks to Resolve System’s enterprise-wide incident response, orchestration and automation platform. Through Resolve, Zayo created a dashboard that automates the routine health checks of disparate systems.

This led to a reduction in 30,000 tickets annually; largely the eradication of false positives and duplicate tickets. The kind of setup for success that Resolve offers, in de-escalating false positive alerts, allows Zayo customers to manage service for themselves; putting the power in its customer’s hands to monitor service cleanly.

Look ma, no hands!

It should be noted a pivotal feature of Resolve, above other platforms for security IR, is that no elevated privileges are required to check on triggers. Resolve doesn’t need a root account, super user or admin tokens to check and validate false positives.

Identifying Red Flags or Red Herrings in Incident Response

With customers in your face and management at your back, it can be difficult to i.d. legitimate red flags or misleading distractions. Add to that a lack of centralized knowledge management and an incident response with dizzying layers of applications; infrastructure; and systems that don’t directly communicate to each other.

Overcoming false positives means adept filtering into the Sec Ops or IT Ops centers appropriately and filtering out duplicitous and erroneous alerts. As Marcus Rebelo, Director of Sales Engineering with Resolve Systems explains, SIR requires a dexterity and perception.

“Risk management of threats and exposed vulnerabilities,” are aspects of security use cases to compare in choosing platforms, Rebelo says. He adds there needs to be tracking and deliverables to “identify and respond to attacks against the organization’s information systems from external threats. This includes monitoring for worms, viruses, denial-of-service, and other similar attack vectors.”

Let’s take a look at several use case scenarios, what the Resolve Platform offers in degrees of automation—from end-to-end to human-guided—and what the solution does for determining red flags or red herrings.

Automated Incident Response … Game On! Flag rules

Scenario 1: Vigilant critical event triggers often derailed by false positives.

What Resolve Automated SIR Offers:

  • Active channel(s) showing only critical incidents. Rules that automatically open and add related artifacts to cases. Critical ability to exclude or blacklist specific patterns or URL’s based on normal operations or misconfigured devices
  • Optional integration with external ticketing system

Wave the flag: The deliverables signal for automations to be human-guided; inviting ad-hoc analysis and response to the generated cases and subsequent funneling of identified false positives to the appropriate teams—be it IT Ops, Service Desk or SOC.

Scenario 2: Intrusions.

What Resolve Automated Security IR Offers: Isolation of actual breaches, while recording and quelling false positives.

Flag down: Automation rules allow for real-deal intrusions to be stopped without further diversion, while keeping track of all pertinent data to be used in a later investigation or review. Before operations kills an intrusion, Resolve can capture it and get all shell history and black box details at the speed of compute.

Scenario 3: Network equipment/UNIX with, let’s say, nine failed logins in a single day.

Ambiguity: Is it a brute force attack or an IT help desk login issue?

What Resolve Automated Security IR Offers:

  • A weekly or daily automation showing long-term trends in account activity plus a rule generating a critical incident when there are X failed logins over some time period greater than a previous time period’s average + a given number of standard deviations (or equivalent), working with big data systems like Splunk
  • A rule generating a critical event for VIP accounts such as administrators
  • The aforementioned rules will include a means of filtering or whitelisting permanent or temporary activity for a specific account or system

Capture the flag: Annotations can point to red-flag alerts as being likely a high instance of false positives for misconfigured devices.

Scenario 4: Disconcerting fluctuations in event volume: from surges to eerie quiet in event noise.

Ambiguity: Have devices just stopped sending data or is there a security issue?

What Resolve Automated Security IR Offers: Daily executed automations for information to data systems showing –

  • Automation results of devices that have suddenly not generated any events that previously were generating events
  • Validation and diagnostics of new devices generating events
  • Devices generating abnormally more or fewer events than the past 24 hour period
  • An optional human guided procedure providing multiple paths to automated operational activities for critical event sources that have stopped sending events

Flag day. This deliverable set allows for human guidance to tune in for false positives in devices curating low event volume every 24 hours.

Scenario 5: Suspicious VPN activity.
What Resolve Automated Security IR Offers:

  • An automation creating a critical incident when VPN logins occur for the same user from different countries within 12 hours
  • An automation for any VPN user logging in remotely without a physical token
  • Automations displaying VPN activity

Fly the flag. Resolve is the only platform that allows for the obvious human aspect of problem-solving to lead, e.g. checking for employee travel and subsequent VPN access attempts, while still incorporating automation for heavy lifting activities.

Scenario 6: A phishing campaign.

What Resolve Automated Security IR Offers:

  • Creation of a critical incident when the exact same file is downloaded from multiple workstations in a blitz; signifying a mass phishing attempt
  • Blacklisting based on a site or filename with the ability to escalate or identify when a proxy is blocking or not-blocking events and checking whitelists during the process

Flag it down. False positives can occur in this scenario, but just like a police checkpoint, all traffic has been flagged down and authorities are critically monitoring everyone coming through. The likelihood of risky business passing through has been halted.

Scenario 7: Phishy domains popping up around the same time as legitimate new domains; potentially rerouting a good volume of interest of customers.

What Resolve Automated Security IR Offers: Integration with an external system

  • Record the registration date for all resolved domain name systems in the entire data stream
  • Generate a critical incident when multiple systems visit a new domain in a short time span
  • Blacklist based on site or filename and escalate or identify when the proxy is blocking or permitting events, also checking against maintained whitelists

Put the flags out! Cyberthreats and Critical IT alerts are deciphered and neutralized in record time when automations of varying degrees allow for the ability to shift left, from SMEs to—now equipped—L1 agents and customer self-service, as in the case of ZAYO.

Read more on Caught in the Riptide of Alert Fatigue: 3 Key Capabilities of a Security Incident Response and Automation Platform

A unified platform for incident resolution for both IT Ops, NOC, and SOC ensure that not only are all potential resolutions exhausted with a uniform and traceable knowledge base but all centers are collaboratively involved.

One leading airline reports some bridge calls are burgeoned with up to 250 people from Sec Ops, IT Ops, Service Desk and Network Operations teams, all trying to make sure he or she is “not it” with respect to the source of the incident or the subject-matter expertise needed to respond.

Resolve Systems has the only incident response and automation platform on the market that integrates with—and within—existing infrastructure, circumventing the OpEx and Operations hassles of “rip and replace.” We are the only vendor that addresses the entire IT and security framework. Our enterprise-wide incident response platform empowers your team to respond to the right ticket, at the right time, for the quickest response possible.

Eliminate the false alarms–and prioritize your alarms with subject matter expertise down-shifted to Level 1 agents–for the quickest incident response application on the market.

Download our latest white paper: Security Incident Response Needs a Unified Platform.


About the Author, Resolve Staffer:

This post was written by one of the awesome contributors on the Resolve team.

Recommended Reads

What Is the State of IT Automation Going into 2023?

What Is the State of IT Automation Going into 2023?

Where is automation heading? Business leaders track the latest trends.

Automate. Innovate. Celebrate.

Automate. Innovate. Celebrate.

Automation is the backbone of the digital enterprise.

Observability and Auto-Remediation

Observability and Auto-Remediation

Learn how Resolve addresses alert overload with auto-remediation