(And by the way, when you meet other organizations claiming to do what Kenna does, ask them to list out their exploit intelligence partners. All of them. And how they integrate that intelligence with your vulnerability scans. Then watch their faces go white as they stutter their way through an answer. It’s a fun game!)

Kenna is proud to introduce our new intelligence partner: Exodus. They deliver unparalleled zero-day data that’s unmatched in the industry.

We believe that Exodus has a unique view of the world; they share Kenna’s belief that it’s imperative to know which corners of your network houses vulnerabilities for which there are no current patches.

Addressing risks from zero days requires you to change your security practice to be faster than the rate of disclosure. It requires you to do research faster than MITRE or NVD and to correlate to assets based on imperfect data. Addressing zero day risks also requires the centralization of decision making. If you’re looking at a vulnerability scan and making decisions about it, then looking at a second scanner and making those same decisions about it, and then a third engineer looks at a threat intel feed and raises his hand – you are in disarray. Conversely, if all of your intel and vulnerability data is in one place, you can make and measure decisions about overall risk.

With Exodus, Kenna continues to offer its customers a detailed knowledge of unpatched vulnerabilities and helps them measure, monitor, and reduce their overall exposure to risk.

Within the Kenna platform, environments that are susceptible to Exodus’ Zero-Day vulnerabilities will be displayed in their primary dashboard, making it easy for users to take appropriate action. If they do experience an issue with a Zero-Day, users will be able to obtain extremely detailed reports and exploit details on the specific vulnerability from Exodus Intelligence.

Learn more about Exodus Intelligence!

The Urgency of Vulnerability Remediation

If security is a race, it’s a sprint. This is particularly true when vulnerabilities are discovered. The race begins, with the good guys figuring out where they’re exposed and how to address the gaps, while the bad guys start building their modes of attack. The problem is that bad guys have an unfair advantage, leveraging automation to wage broad-based, indiscriminate attacks.

Security teams are falling behind. The reality is that vulnerabilities are most likely to be exploited within 40 to 60 days of their release, while security teams typically take 100 to 120 days to remediate vulnerabilities—according to Kenna Security’s proprietary research.

This is a race that also calls for precision. In an enterprise, security teams have tens of thousands of vulnerabilities to remediate. There’s no way they’ll address them all ever—let alone anywhere close to that 60-day window. Therefore, it’s critical that administrators don’t waste time remediating vulnerabilities that aren’t being exploited. Security staff needs to identify the vulnerabilities that pose the biggest risk—and the key is knowing which those are.

The Emergence of Vulnerability Marketing

As security teams struggle to stay current with the threat landscape and determine how to prioritize their efforts, they have to wade through a dizzying amount of news, hype, opinions, and other noise. In recent years, the noise volumes have continued to be amplified. For that you can blame a lot of factors, but one that may not come to immediately to mind is Heartbleed.

Back when Heartbleed was announced, a new precedent was set. Here was a vulnerability with its own marketing engine, featuring a logo, a web site, and more. Heartbleed was a critical vulnerability affecting the most popular open source cryptographic protocol, one relied upon by millions of sites. This was a vulnerability worthy of widespread attention, and using marketing to increase awareness was a good thing.

The Problem: Distinguishing Between Hype and Risk

The problem is that, since that time, around ten vulnerabilities have received similar “star” treatment. Some were critical and worthy of attention, others weren’t. It’s almost starting to feel like this kind of promotion is the only way to get folks to pay any attention to vulnerabilities.  (The recent ImageTragick campaign was a great illustration of this phenomenon.)

When security teams are relying on media, forums, and other public resources to assess their vulnerabilities, they’re in trouble. The process is simply way too arbitrary. Clever marketing can hype a vulnerability, and get staff chasing a vulnerability that doesn’t pose any real danger. At the same time, the increased noise is very likely to drown out the real risks that should be getting addressed.

Consider just a couple examples of some non-marketed vulnerabilities that are still being exploited:

• CVE-2010-3055. This vulnerability was exploited 121,000 times in one year. It allows attackers to run malicious code in phpmyadmin, which is used to run millions of sites worldwide. This only received a Common Vulnerability Scoring System (CVSS) score of 7.5 out of 10, so has remained under the radar of many teams—but it continues to be exploited.
• CVE-2002-0649. First discovered in January 2003, this exploit affects Microsoft SQL Server and Microsoft Desktop Engine. One may think that given how long ago it was discovered that this is a vulnerability we could forget—but one would be wrong. Many thousands of exploits continue to be seen.

Publishing these kinds of exploits won’t get you press or notoriety, so much as a stifled yawn. If you’re waiting for some high profile news stories to appear on these types of threats, you will be waiting a while. A vulnerability shouldn’t have to be new and buzz-worthy to get attention. The quality of logo design or a firm’s social marketing prowess shouldn’t determine which vulnerabilities get addressed and which don’t.

This graph shows the significant delta between successful hits on “Logo’d vulns” versus ones that go mostly unnoticed. It’s the quiet ones that have remote code execution, an exploit, and an active Internet breach.

The Top 4 Factors to Consider

These recent developments point to a bigger problem: Many security teams lack an objective, consistent way to find out about vulnerabilities, and to determine how to prioritize them. Security teams need a way to be alerted to threats and prioritize their efforts based on factors that really matter.

Based on the extensive intelligence Kenna has collected, we have identified common factors that distinguish between vulnerabilities that are critical, and likely to pose real threats to businesses, and those that aren’t. In the following sections, I’ll outline some of the most important factors. These factors are all important; they’re not listed in any kind of priority order. Spoiler alert: The quality of a vulnerability’s logo design, the catchiness of its name, and the number of its Twitter followers are nowhere to be found.

#1. Allows Remote Code Execution

Remote code execution is ultimately what the bad guys are after. Once bad guys have established a way to run their code on a remote system, they inflict all kinds of chaos, whether they want to set up bot networks, steal data, or infiltrate networks. If a vulnerability doesn’t permit remote code execution, it’s one that will typically pose less risk than a vulnerability that does.

#2. Has a module in Metasploit

Metasploit has emerged as the de facto standard for exploit development. Many enterprise security teams and security firms use Metasploit to do penetration testing of an organization’s defenses and identify weaknesses. The problem is that the bad guys can also use Metasploit, and these aren’t just tests—they’re real attacks. When modules appear in Metasploit, you can be assured that a lot of bad guys are, or will soon be, leveraging them in their attacks.

#3. Is network accessible

Whether or not a vulnerability is network accessible can play major role in the severity of the threat and the likelihood of being exploited. Today’s bad guys are all about automation and scale in waging their attacks. The only way to achieve these ends is through network-accessible vulnerabilities that form the basis of botnets, command-and-control communications, and so on.

#4. Is included in Exploit Database

The Exploit Database is a comprehensive repository of exploits and proof-of-concept attacks. Like Metasploit, Exploit Database is invaluable for good guys and bad guys alike. Until a vulnerability appears in the Exploit Database, it remains less likely to emerge as a significant, broad-based threat for organizations.

How Kenna Can Help

Assessing all these factors represents one of the most critical efforts for security teams. However, it also represents a significant effort that has to be sustained. If security teams are going to address the growing gap being created by automated, broad-based attacks, they must get actionable intelligence and they must respond quickly and at scale. This means streamlining and accelerating efforts wherever possible. Quite simply, security teams need to fight automation with automation.

With Kenna solutions, that’s exactly what security teams can do. With Kenna, security teams don’t have to do all the work of manually collecting and analyzing vulnerabilities that get discovered. Kenna solutions automate and streamline the intelligence gathering process. These solutions make it easy for security administrators to distinguish between what’s really being exploited versus what’s being effectively hyped. With Kenna, security teams can more intelligently manage efforts, including remediation and security investments, so they can apply their efforts to the activities that matter most: those that significantly reduce risk.

Summary

Reading security news is a great way to keep tabs on what’s happening, but it’s not a suitable basis for formulating your remediation strategies. Don’t let the cleverness of a vulnerability’s marketing campaign dictate how you prioritize your efforts. Make sure you’re looking at the most critical aspects that figure to predict how much danger a vulnerability really poses. When you do that, you can align your limited time and resources with those efforts that will yield the biggest dividends in reducing risk to your organization.

Here’s a hint: it’s not just about the scanner anymore. It’s about automating the tedious and error-prone processes of prioritization and reporting. Read the full infographic below:

I see it all the time: Security teams live by their spreadsheets. They have lists of vulnerabilities. They stack rank them by severity, start with the most critical, and commence to work through the list.

The problem? All this effort is being expended without a clear understanding of real business risk. Therefore, critical questions remain unanswerable: Where is risk now and where was it before? Where does the risk level need to be? Are we making progress?

In this paper, I’ll detail some common problems I’ve seen security teams encounter when they focus on vulnerability remediation. These cases provide a very vivid illustration of the limitations and risks of focusing on vulnerability counts—and they underscore how much better things can be for organizations that are taking a holistic risk measurement approach.

Problem #1: Focusing on the Wrong Metrics

The security team at Acme Corporation runs their monthly vulnerability scan report and sees the environment contains 500,000 vulnerabilities. Through an “all-hands-on-deck” effort of late nights and concerted focus, they close out 50,000 vulnerabilities before the next month is out. Great work, right? Not so much. The next monthly report comes out and they see there are now 525,000 vulnerabilities.

What’s next, more coffee and even later nights? The reality is that without making significant additions to headcount, it will most likely be impossible for Acme to make a meaningful dent in vulnerability numbers each month.

While these numbers are depressing in their own right, I’d argue they’re the least of the Acme’s problems. Ultimately, the focus is all wrong. Even if they could make a significant dent in vulnerability counts, a critical question remains: Has risk been reduced at all, and, if so, to what degree?

Contrast this reality with Acme after they start taking a risk measurement approach.

The team starts with a focus on assets, and works back from them to identify threats and determine how to guard against them. Every day, their efforts are dedicated to reducing risk, and they’re clear on the steps that will be most effective in realizing that objective. They may still have 500,000 vulnerabilities open. Even if total vulnerabilities rise, risk will tend to be going down because staff is focusing on the most critical areas of exposure. The team can prioritize efforts and track progress against meaningful objectives. Ultimately, staff can be more effective at what matters most: reducing risk to the business.

Problem #2: Ignoring Assets

The reality for any security professional is that there are always more tasks than hours in the day. The trick is trying to figure out which tasks to prioritize, and which to ignore.

Historically, when staff members were weighing priorities, not much attention was given to Acme’s public facing web site, which delivers publicly available technical resources to electronics consumers. While Acme requires visitors to register to access the site, they don’t charge site users or collect any payment information.

Acme’s leadership team doesn’t feel that their site and assets are likely targets, so they don’t want to allocate a lot of their limited IT budget and staff time to securing these online resources. Acme’s security team had plenty of other tasks to juggle, so they didn’t argue.

The problem is that given their site’s popularity, they’ve ultimately amassed a registered user base of 100 million, including email addresses and other personal information. Further, their site is routinely visited by thousands of unique visitors every day. Because they’re focusing on vulnerability remediation, the security team and executive leadership haven’t recognized the value of the assets associated with their online properties.

While the site’s value may be overlooked internally, it represents an appealing target for those who are constantly looking for user machines to compromise. The bad guys may launch a series of exploits to gain access to user emails, so they can launch a series of phishing attacks. Or, lured by the large volume of visitors, the criminals may exploit a persistent cross-site scripting vulnerability on Acme’s web servers that enables them to infect site visitors’ systems. Once they’ve infected enough machines, they may start creating botnets to launch DDoS attacks. Staff members don’t take the steps needed to guard against these types of attacks, and they’re incapable of stopping them when they happen.

By contrast, when Acme’s security team takes a risk measurement approach, they look at risk holistically, employing techniques like threat modeling. That means factoring in not only vulnerabilities, but Acme’s assets, and the types of threats they’re exposed to. Through threat modeling, they’ll do an objective analysis of the value that data, systems, and users may have to an attacker. They’ll also assess how sites and applications are used, the number of users, the types of attacks being waged against other organizations, critical vulnerabilities discovered on specific systems, and a number of other factors.

Through risk measurement, the security team can gain a much more complete picture of the most critical threats facing their business. Perhaps even more importantly, they’ll be able to gather objective analysis of threats and overall risk, so they can report effectively to executive leadership, and ensure investments are aligned with security priorities.

Problem #3: Investing Time and Money on the Wrong Efforts

After running a scan, John, an administrator at Acme, finds there are 10 vulnerabilities on a given system: one critical, two high, and seven low. Because John and his management team have a vulnerability-count mindset, he focuses on closing those seven low-priority issues and claims victory to his superiors: Vulnerabilities were reduced by 70 percent.

The problem is that John may not have reduced risk much or at all—and a lot of time, effort, and money went into that lack of accomplishment.

Once the team at Acme employs a risk measurement approach, things change substantially. By factoring in exploit threat intelligence, for example, they may find that, of the 10 vulnerabilities referenced earlier, one of the high-priority vulnerabilities is the one that’s currently being exploited most frequently by a sophisticated adversary. By closing that one vulnerability, John may reduce total risk by 70%, while spending far less time and money on remediation.

Conclusion: The Risk Measurement Payoff

By moving from a focus on vulnerability remediation to risk measurement, organizations can reduce business exposure in two ways. First, they can reduce the likelihood of a breach. Second, they can minimize the potential impact of a breach should one occur.

I once heard a CISO say, “We can’t work any harder, we have to work smarter.” Fundamentally, risk measurement provides a way for security teams to work smarter. They can focus their time, budget, and resources on what matters most: reducing risk. Risk measurement also provides teams with a centralized way to accumulate, analyze, and report on risk, which helps significantly improve operational efficiency.

Risk Measurement Can Take a Lot of Effort—Kenna Can Help

While it’s easy to see how risk measurement can help, that doesn’t make it easy to do. A lot of intelligence needs to be correlated to do risk measurement right, and Kenna can help. Kenna automates much of the effort associated with aggregating and parsing intelligence, so security teams can much more easily measure risk and determine the most significant and efficient ways to reduce risk. By combining Kenna with best practice threat modeling approaches, organizations can employ their resources most intelligently to reduce the likelihood and potential impact of breaches.

The Kenna platform processes billions of pieces of vulnerability and exploit data daily for our customers, helping contextualize vulnerabilities so that security teams know what to prioritize and fix in their own environment. The data we submitted to the Verizon top 10 only used a raw subset of 3rd party exploitation data without taking any of the contextual data or our prioritization algorithms into consideration. As one of our customers constantly reminds me, “We can’t work harder anymore than we do today – we have to work smarter – and that is what your platform allows us to do.”

Looking at the much-discussed FREAK vulnerability as an example, if we had actually run the data through our platform and algorithms, it would not have risen to the level of a significant vulnerability. The Kenna platform is designed to ensure that our customers don’t prioritize a patch that could be a false positive or outlier by taking into account many variables including: volume and velocity of the exploit, exploit availability, weaponization of the exploit, whether or not that exploit has been observed as part of a greater campaign, relative priority of the asset on which the vulnerabilities sit, and over a dozen external sources. We looked at FREAK within the Kenna platform and saw that CVE: 2015-0204 had a Kenna score of 25.0372 (out of 100). This is nowhere close to even a top 10,000 vulnerability or even in the top 70% of all vulnerabilities.

I have always believed that you need to be clear about and uphold your values and this experience only underscores this belief. We at Kenna deeply value integrity in all of its forms, but especially of our data as it helps our customers “work smarter.” As we clearly did not exhibit that integrity with the top 10 results, we felt it was important to set the record straight.

Karim Toubba
CEO – Kenna Security

Recently, Adrian Sanabria from 451Research called me to question some of the assumptions I used to write the 2016 Verizon DBIR Vulnerability section. We had a two-hour chat and some follow up emails, and he led me to think of a new way to look at this data. This is absolutely indispensable when we defenders are working together against a sentient attacker. It’s also thrilling! Where else does one get to collaborate with competitors, researchers and clients alike as part of of their day to day job?

Adrian’s big insight was that the top 10 list of vulnerabilities is confusing because the applicability to the enterprise is lost on these arcane, working exploits, and the gap between peoples’ expectations and reality is creating some friction. We had an excellent offline discussion in which he dove deeply into the assumptions of my work, asked thoughtful, deep questions in private, and together, we came up with a better metric for generating a top 10 vulnerabilities list.

To address these issues, I scaled the total successful exploitation count for every vulnerability in 2015 by the number of observed occurrences of that vulnerability in Kenna’s aggregate dataset. Sifting through 265 million vulnerabilities gives us a top 10 list perhaps more in line with what was expected – but equally unexpected! The takeaway here is that datasets like the one explored in the DBIR might be noisy, might have false positives and the like, but carefully applied to your enterprise the additional context successful exploitation data lends to vulnerability management is priceless. Generating signal is science. So here’s a new top 10 list (biased because occurrence rates are measured by Kenna Security customers, a very convenient convenience sample).

Prevalence Scaled Top 10 Vulnerabilities

1. CVE-2002-0013 – Vulnerabilities in the SNMPv1 request handling of a large number of SNMP implementations allow remote attackers to cause a denial of service or gain privileges via (1) GetRequest, (2) GetNextRequest, and (3) SetRequest messages
2. CVE-2002-0012 – Vulnerabilities in a large number of SNMP implementations allow remote attackers to cause a denial of service or gain privileges via SNMPv1 trap handling
3. CVE-2015-0204 – The ssl3_get_key_exchange function in s3_clnt.c in OpenSSL before 0.9.8zd, 1.0.0 before 1.0.0p, and 1.0.1 before 1.0.1k allows remote SSL servers to conduct RSA-to-EXPORT_RSA downgrade attacks and facilitate brute-force decryption by offering a weak ephemeral RSA key in a noncompliant role, related to the “FREAK” issue. NOTE: the scope of this CVE is only client code based on OpenSSL, not EXPORT_RSA issues associated with servers or other TLS implementations.
4. CVE-2001-0540 – Memory leak in Terminal servers in Windows NT and Windows 2000 allows remote attackers to cause a denial of service (memory exhaustion) via a large number of malformed Remote Desktop Protocol (RDP) requests to port 3389.
5. CVE-2015-1637 – Schannel (aka Secure Channel) in Microsoft Windows Server 2003 SP2, Windows Vista SP2, Windows Server 2008 SP2 and R2 SP1, Windows 7 SP1, Windows 8, Windows 8.1, Windows Server 2012 Gold and R2, and Windows RT Gold and 8.1 does not properly restrict TLS state transitions, which makes it easier for remote attackers to conduct cipher-downgrade attacks to EXPORT_RSA ciphers via crafted TLS traffic, related to the “FREAK” issue, a different vulnerability than CVE-2015-0204 and CVE-2015-1067.
6. CVE-2012-0152 – The Remote Desktop Protocol (RDP) service in Microsoft Windows Server 2008 R2 and R2 SP1 and Windows 7 Gold and SP1 allows remote attackers to cause a denial of service (application hang) via a series of crafted packets, aka “Terminal Server Denial of Service Vulnerability.”
7. CVE-2001-0877 – Universal Plug and Play (UPnP) on Windows 98, 98SE, ME, and XP allows remote attackers to cause a denial of service via (1) a spoofed SSDP advertisement that causes the client to connect to a service on another machine that generates a large amount of traffic (e.g., chargen), or (2) via a spoofed SSDP announcement to broadcast or multicast addresses, which could cause all UPnP clients to send traffic to a single target system.
8. CVE-2001-0876 – Buffer overflow in Universal Plug and Play (UPnP) on Windows 98, 98SE, ME, and XP allows remote attackers to execute arbitrary code via a NOTIFY directive with a long Location URL.
9. CVE-2013-0229 – The ProcessSSDPRequest function in minissdp.c in the SSDP handler in MiniUPnP MiniUPnPd before 1.4 allows remote attackers to cause a denial of service (service crash) via a crafted request that triggers a buffer over-read.
10. CVE-2014-0160 – The (1) TLS and (2) DTLS implementations in OpenSSL 1.0.1 before 1.0.1g do not properly handle Heartbeat Extension packets, which allows remote attackers to obtain sensitive information from process memory via crafted packets that trigger a buffer over-read, as demonstrated by reading private keys, related to d1_both.c and t1_lib.c, aka the Heartbleed bug.

What we set out to build at Kenna Security is a new way of thinking about information security vulnerabilities – a framework for efficiently measuring remediation, a way to stay ahead of attackers by making sure that every action was a useful one.

At first it seemed like an impossible task, precisely because of the issues outlined in @attritionorg’s incredibly thorough and well thought out criticism of the Verizon DBIR’s vulnerability chapter. On the whole, Brian is correct: IDS alerts generate a ton of false positives, vulnerability scanners often don’t revisit signatures, CVE is not a complete list of vulnerability definitions.

But those are just the trees, and we’ll get to them later. The forest is that this somewhat stark status quo is exactly the situation faced by thousands of enterprise practitioners, who are armed with nothing but their vulnerability scans and some logs, and must wade through millions of vulnerabilities and determine what must be done next. We seek to combine these datasets, in some inventive ways – not to say “this somewhat noisy data indicates you must fix an FTP vulnerability or perish”, but rather to use exploitation data to add context to vulnerability scans, and to confirm some assumptions about automated attacks.

But most importantly, we seek to give the reader of the Verizon DBIR cold hard data to take back to management and be able to say “Yes I know it’s patch Tuesday, but today I shall fix a CVE from 2006, because it’s likely actively exploited, and 273 of our servers are vulnerable to it”. That is the reason the Verizon DBIR is useful and esteemed. Insights generated from data, no matter how counter-intuitive the insight, is the whole of why data science exists and can help security practitioners.

Dan Geer and Jay Jacobs’ article on how to properly describe convenience samples (make no mistake, this is one of them), borrows a few excellent ideas from medicine and lays out a few guidelines, so I will follow them here (better late than never):

The data used by the Verizon DBIR Vulnerability section is comprised of two datasets.

The first is a convenience sample that includes 2,442,792 assets (defined as: workstations, servers, databases, ips, mobile devices, etc) and 264,912,235 vulnerabilities associated to those assets. The vulnerabilities are generated by 8 different scanners, they are: Beyond Security, Tripwire, McAfee VM, Qualys, Tenable, BeyondTrust, Rapid7, and OpenVAS . This dataset is used in determining remediation rates and the normalized open rate of vulnerabilities.

The second is a convenience sample that includes 3,615,706,022 successful exploitation events which all take place in 2015 which come from partners such as Alienvault’s Open Threat Exchange.

Please note the methodology of data collection: Successful Exploitation is defined as one successful technical exploitation of a vulnerability on one machine at a particular timestamp. The event is defined as: 1. An asset has a known CVE open. 2. An attack come in that matches the signature for that CVE on that asset and 3. One or more IOCs are detected/correlated post attack. It is not necessarily a loss of a data, or even root on a machine. It is just the successful use of whatever condition is outlined in a CVE. If any readers would like to see a sample of the dataset, feel free to each out to me at my Kenna Security email, michael@kennasecurity.com. Below are descriptive statistics for every CVE in the original top 5. The data comes from internally instrumented dashboards which will update live as new data rolls in (every hour).

1. CVE-2015-1637 http://www.stathat.com/s/4813HmgKhyPk
2. CVE-2015-0204 http://www.stathat.com/s/YSHQp2H6OAin
3. CVE-2003-0818 http://www.stathat.com/s/4qHxrmPDWum3
4. CVE-2002-1054 http://www.stathat.com/s/8TzXxTnyYyJ3
5. CVE-2002-0126 http://www.stathat.com/s/14wGpFQFpe6w 

Enterprises use vulnerability scanners to manage vulnerabilities, the lowest-hanging fruit is to answer the question “Of these vulnerabilities, where should I focus my remediation efforts?”. In fact, the entire vulnerabilities section of this year’s DBIR focuses on what can be done given the status quo of vulnerability management, not what should be done in a perfect world. The underlying distributions indicate that 1. Attackers target old vulnerabilities often 2. Attackers automate their campaigns and spray across the internet. The implications can then be used to formula a strategy that is more effective. Essentially, patch Tuesday rolls along. Everyone scrambles to fix the new MS vulnerabilities. The data says “No. Be better.”

Data is your voice of reason.

Our point is not “If you don’t have BlackMoon FTP you’re safe”. Our point is that fat tailed statistical distributions necessitate a different approach to vulnerability management. It would be a shame if we lost the forest for the exploit signatures.

Must Have #4: Know Your Resources

Once you have a good handle on your business, your assets, and what security risks are currently affecting your environment, you’ll need to understand your resources. What do you have at your disposal to eliminate risks going forward—including people, money, time or a combination thereof?

If you have identified what resources you have at your disposal, you can set reasonable or stretch goals to eliminate security risk targeting the environments and processes that matter most. You, of course, could turn this approach on its head and use the first 3 “must haves” to identify and budget the appropriate resources to manage risk down to an acceptable level for your business.

What areas are most crucial for risk reduction within your business? What security risks are you carrying within those areas? What types and how many resources do you need to remove risk?

“Risk” here should be thought of as uncertainty, specifically within a security context. By knowing your resources and targeting the greatest “bang for your buck” risks, you’ll be able to reduce risk in a very efficient manner.

Some useful metrics here include:

1. Budget spent on security remediation
2. Risk carried above tolerance level
3. Hours per security solution

Keep in mind: When you’re tracking your progress in reducing risk, it’s important not to fall into the trap of simply counting vulnerabilities. While closed vulnerabilities is a metric that all teams will want to continue to track, what’s more important is the ability to reduce overall risk and how to effectively do that with the resources at hand. It may even be possible that as the most critical vulnerabilities are closed—but a relatively small number—the trend line of overall vulnerabilities may rise while the risk line goes down.

Must Have #5: Know Your Direction

You’re now in a state where you understand where your assets are, how to discover new assets as they pop up, what risk these assets carry based on your business and your security weaknesses, and what resources you have at your disposal to reduce that risk in an efficient manner.

It’s now time to continuously measure these metrics to understand your direction and set meaningful goals to reduce risk over time while continuing to support business objectives. As baselines are established, you can then work with the organization to target the areas of risk that are not within an acceptable range.

With knowledge of the resources you have at your disposal, you can quantitatively demonstrate what a reasonable risk reduction goal could be or make a case for additional resources based on your organization’s risk tolerance.

Some useful metrics here include:

1. Risk reduction by asset group over time
2. Risk goal by asset group
3. Cumulative risk accepted over time

And finally, it should be possible to preview how much your risk will be adjusted by any activity. This ensures that you are fixing the most critical issues first and spending your resources wisely.

Summin’ Up…

Vulnerability Management in a function that needs to be driven by the right metrics, with the goal of identifying and reducing the organization’s overall exposure to risk. Having the appropriate set of numbers, metrics, and measurements will facilitate that process.

The ultimate objective is to have an accurate overall picture of risk—both in terms of overall asset counts, the threats particular to the business, the resources you have available, and the trend line of your overall risk. In doing so, vulnerability management becomes as clear and precise as possible in an environment where the threats themselves are murky, ever-changing, and increasingly difficult to identify.

Must-Have #2: Know Your Business

In order to understand the most pertinent threats and measure the likelihood of exploits, you really need to understand these factors within the context of your business. A great way to apply this knowledge to security is through threat modeling. There are numerous methodologies for threat modeling out there, ranging from the heavyweight to the “back of the napkin.”

But don’t worry about starting with something highly sophisticated: there are significant advantages to be realized by just going from nothing to something. By knowing even some of the most basic attributes about your business and applications, you can begin to understand what are the most likely threats. Some simple examples include:

1. What broad metadata do you have about the organization? Your industry vertical, size, and geography all play a role here.
2. What information is processed by your application or assets, and what are the value and/or regulations around such information?
3. How many people use an application? This is an often overlooked but critical attribute.
4. What and where are the key assets to your business? Are there critical controls for these assets to protect confidentiality, integrity, or availability?
5. Who are your adversaries and what are their capabilities? Occam’s Razor applies here.

Some useful metrics here include:

1. System Susceptibility
   1. Value to Attackers
   2. Vulnerabilities
2. Time to Compromise (Hacker economics): How long would it take to compromise any of the key controls for these assets and applications?
3. Threat Accessibility
   1. Access Points and Attack Surface
4. Threat Capability
   1. Tools
   2. Resources

Does your threat model include Alexa ratings? As an example of this, take two applications. One is used by your internal employees for HR and contains sensitive information such as social security numbers, health care information, etc. The other is a public application that contains no sensitive information and all data is public.

If you don’t evaluate the size of your user base, it’s obvious that the application processing sensitive data far outweighs the risk of the public application. However, if the public application has 100 million users—and these users can be attacked through a persistent cross-site scripting vulnerability in the application allowing a malicious user to attack 100 million users—does that change things?

Must-Have #3: Know Your Risk

Counting vulnerabilities and relying on static scores no longer works; these methods don’t account for the fact that threats change constantly and there needs to be a tried-and-true methodology in place to measure real security risk. If you build your process on top of a broken risk model, you end up in a riskier position faster. Taking a risk-based approach over quantities and an “effort complete” method is a must.

In order to understand your risk, at a minimum you’ll need a handle on both likelihood and impact. There are many factors that go into such a methodology. Some questions to ask include:

1. Asset metadata: Fed by the first two “must haves” areas, do you understand who owns the asset, what the function of the asset is, and how it’s used? What’s the impact of losing the confidentiality, integrity, or availability of the asset? Which is the most important based on its function?
2. Vulnerabilities: What are the weaknesses and vulnerabilities tied to this asset or group of assets? How easy or difficult is it to exploit these weaknesses?
3. Threats: What are the threats associated with the security holes as well as to your business? How skilled is your adversary and what skills are required to exploit your weaknesses? How prevalent are these vulnerabilities being exploited in the wild? Are you likely to be hit by a “drive by”?

Most importantly, there should be a single score that unifies the entire environment, based on real-time exposure to risk. From there, it’s possible to move down into other important asset groups and categories.

You also need to be able to track your progress over time. Think of exposure to risk like a stock report. What was your exposure last week—and last month? Has the risk line trended down or up? What’s your “high” and “low” point over the past 52-weeks?

Showing your team’s ability to reduce exposure to risk over time is a critical component of arguing for a team’s efficiency, ability and—at the end of each year—the need for additional budget and headcount.

Some useful metrics here include:

1. Risk by asset group both current and trending over time
2. Mean-time-to risk reduction where risk reduction is a target or goal
3. Time to remediate high risks broken down by asset groups

We still have two more Must-Haves coming up. Stay tuned for Part 3…

The rising cadence of automated attacks means that security teams need to strive to make their own practices as precise and metric-driven as possible. Pouring through spreadsheets and creating 500-page PDFs is no longer enough to ensure that critical vulnerabilities are remediated in time. But what’s the best way to ensure that the right metrics are applied to the practice of vulnerability management—a security function that has occasionally been seen as directionless in the past?

Here are a few key areas where you’ll need to apply Must-Haves Metrics for Vulnerability Management:

Know Your Assets: Do you know where all your assets and applications are? What is your current assessment coverage? How do you discover new assets?

Know Your Business: Are you performing threat modeling? What threats exist to your business? Are you a target?

Know Your Risk: Where are your security weaknesses and vulnerabilities, and which ones are the most likely to be exploited? How do you determine likelihood and impact?

Know Your Resources: What can you get done with the resources you have? Are you accounting for budget, time, and people?

Know Your Direction: Are you getting better or worse over time? Given the other “must haves” above, what is an achievable goal for risk reduction?

The reality is, the old methods of vulnerability management—using spreadsheets and counting vulnerabilities—still have their place. But for a new world of rising threats and attacks, a new set of metrics is necessary in order to keep pace with the rising cadence of critical vulnerabilities.

Must-Have #1: Know Your Assets

Before you can even begin to asses your security risk or posture, you first need to know what you have. This includes all of your assets—whether in the data center, on your corporate network, as part of remote access, or as part of your applications.

Of course, that’s easier said that done; knowing where ALL of your assets are for any sizeable organization is a daunting task. Identifying 100% of your assets is often dismissed by practitioners as something only vendors say is possible (who’ve never had to do it themselves).

Rather than writing this off as an impossible task, though, treat this objective as a metric with specific goals and progress tracking. There’s a certain Rumsfieldian aspect to asset tracking in that there are “known knowns, known unknowns, and unknown unknowns” on your networks and managed by your organization.

In order to manage and measure this metric, you’ll need an automated discovery process. Starting from the outside in, you’ll need to understand your DNS and WHOIS records. What IP address ranges and domains do you own? What ports, applications, and services are running on them? What is your process for discovering new assets, services, DNS records—and is the process automated? How do you feed these assets into your assessment and scanning processes,

Some useful metrics here include:

1. External scanner coverage (known assets/scanned assets)
2. Internal scanner coverage (known assets/scanned assets)
3. Time to discover (lower is better)

More metrics to come in the next post…

But with the introduction of our new reports, Kenna becomes something else: a security analytics platform that helps organizations measure, monitor, and track their overall exposure to risk. By helping direct the conversation along the lines of risk–as opposed to, say, the number of CVSS-ranked 8s, 9s, or 10s that exist in an environment–Kenna allows its customers to center the conversation around a single metric for risk that even the non-technical business person can understand.

Kenna’s new reporting enables you to:

Track your progress like a stock report. What was your highest and lowest risk score over time? What’s your exposure to risk right this second?

Measure Your Risk by Asset Group or Vulns. Look at associated risk in both asset groups and vulnerabilities–then immediately drill down to see what’s at the highest risk.

Track the vuln numbers that matter most. Even though Kenna keeps your focus on risk, it still gives you the vulnerability numbers that your stakeholders need–including total number of closed vulns, vulnerability aging, mean time to remediate, and more.

Know the work you have left to do. Kenna’s unique “fixes” feature tells you not only what vulnerabilities you have, but what patch will close them. You can report on the effectiveness of our Fixes by seeing how many you still need to implement and the number of CVEs affected by them.

Custom Ranges Take Snapshots of Risk. Track your progress in reducing risk using whatever date range you want. See how effective you’ve been in over time, and prove the impact of your teams’ remediation effort.

Try it for yourself here.