Incident Response and the Usual Suspects

Like many in the ITSM and SIR spaces, being overrun with pricey, lengthy and unnecessary incident escalation and response is all-too familiar. Across silos—Security, Network, IT Operations and Service Desk—it’s the same cast, different characters. Incidents rob the entire enterprise of time and money.

In fact, if incidents resembled a Hollywood-movie heist, here’s a look at the typical conspirators.

Sec Ops, IT Ops, Service Desk and NOC teams are all frequented by the Usual Suspects, the same challenges to effective incident response.

The Intimidator

This character is usually the one hostages see first. The muscle with weapons or the smoke bomb, who overwhelms security and besieges bystanders into submission.

Not unlike IT Ops, Network Ops or Service Desk day-to-day, alerts smother Level 1 agents in what is commonly referred to as a “sea of red” within popular ticketing systems: One intimidating list of “critical” alerts; or event noise that is so blaring, some incidents are missed and operational inefficiencies get in the way.

Incident response analysts have estimated:

• 32 percent of incidents are escalated from L1 to L2/L3; which translates to incidents spiking $100,000 in (salary) cost to resolution up the ladder
• Up to 10 percent of incidents are a good fit for closed-loop automation
• More than 90 percent of incident types can be accelerated using human-guided automation

Resolve Systems has the only platform intentionally built for customizable incident resolution and empowering operators to work efficiently beyond the intimidating “sea of red.” It’s the only enterprise-wide solution built to break down silos across Security Operations, Network Operations and IT Operations, offering both end-to-end and human-guided automations, helping teams shift left instead of escalating and multiplying the cost of response by passing issues to higher—more expensive—resources within the team.

Read more: Shift Left: The NEW ‘Moonwalk’ – or – An IT Strategy your CIO needs to be talking about?

The Diversion

In the movies, this is where the staged outburst or scantily clad vixen creates a scene to take watchful eyes off of access routes and gain entry to “the goods.” False alarms in operations do the same thing.

Industry experts surmise 25-40 percent of tickets are “problem not found.”

For one Global 100 Financial Services Institution, Resolve shaved off 170,000 incidents per year by reducing man hours spent on false positives via automating health checks and security incident response.

Other common diversions slowing down effective incident response are data residing in multiple systems: dispatch, alarm, trouble ticket and inventory for IT Ops, as an example. For SOC, often a single incident demands multiple departments having to carve out and sync time slots for the dreaded bridge calls, necessitating various approvals and SME input from coders, engineers and more.

Resolve has been called the “orchestrator of all orchestrators” in that:

• We provide unified process orchestration and automation for faster incident response—integrating with an enterprise’s existing infrastructure, applications and systems across silos without abrupt “rip and replace”
• The Platform bridges process, productivity and access gaps between Network Operations, Security & IT Operations teams—circumventing time-guzzling “war-room” calls and monopolizing L2 and L3 experts

Resolve Systems provides agents with human-guided automations, real-time incident collaboration, and the ability to both partially and fully automate processes.

The Getaway Driver

This player in a heist may not get their hands dirty on the inside but they are just as guilty as the other thieves in the eyes of the law. Event management accosts the valuable time of subject matter experts; it’s downright highway robbery.

Any L2 or L3 engineer will tell, they “want to be working on innovative projects that are going to drive new revenue or building out automations and best practices for the others on the [escalations] staircase to use. Their time is best utilized this way, so let’s give them the time and the ability to do it,” as our previous article explains.

NOC L2 and L3s: Spend the majority of their time in alert or event management software surveilling alerts and determining which are worth responding to. They’re only involved on the outskirts of planning for new applications and building.

DevOps: L3 programmers develop digestible solutions to be supported and maintained once in production. Hard to do that when mundane tasks obscure. When Resolve captures SME knowledge in pre-built, no code, automations for simpler one-off solutions, L1s can handle smaller tasks and L3s can be freed up to create elegant programmatic solutions and frameworks.

SecOps & IT Ops: Speed is the name of the game for IT Ops and SecOps. The can best flex their muscle in leveraging 3rd party vendor components and solutions; and if ITSM or SIR bridge calls don’t drive off with their productivity time, L2s and L3s can proactively monitor and creatively troubleshoot issues.

Without tedious, redundant and malicious events diverting their time, subject-matter-experts can focus on innovating infrastructure, the overall health of the organization versus reactively putting out fires.

Resolve’s automation builder empowers programmers and engineers to easily build out their own automation flows from scratch—without a single line of code—by using our repository of automation templates.

The Mouth

It’s often quoted “perception is reality” and no matter how polished your enterprise is, if your customers have issues with no answers, hyperextended mean-time-to-recovery is correlated to resounding dissatisfaction. No matter how many millions are allocated toward marketing, customer satisfaction remains the most influential mouthpiece affirming or holding hostage your enterprise’s brand.

A recent Total Economic Impact study of Resolve, conducted by Forrester Research, analyzed two of Resolve Systems’ customers and calculated the “average alarm acknowledgment time went down from 1,889 minutes (31 hours) before the Resolve Systems’ solution to below 1 minute with the solution.”

According to the meta analysis, Resolve’s accelerated incident response through automation created significant increases in customer satisfaction and employee morale. Other Resolve Systems customers relay similar success:

• Columbus communications shaved MTTI from six minutes to four seconds with Resolve Platform
• Virgin Media dissolved 6,500 monthly customer calls using automated incident response
• After shifting repetitive tasks to human-guided automation, Charter restored 30 percent productivity to IT Support staff; which, for an eight hour day translates to 2.5 constructive hours back in the hands of the team

The Response

In a movie-worthy heist, there is often a clever and highly decorated good guy who comes in, reassures the authorities and single-handedly saves the day, outsmarting the conspirators at their own game.

Resolve Systems is—and has long been—that response: accelerating, automating and remediating incident response, across the enterprise for Security, IT Operations, Network Operations and Service Desk teams.

To stay in the conversation on what other “usual suspects” are holding up effective incident response, as well as the best practices to overcome them, RSVP for the upcoming Incident Resolution Summit, sponsored by Resolve Systems.

For insider detail on top companies and their successes with Resolve, click here.