Are you really remediating all the discovered vulnerabilities?

According to an article published by PcMag, LastPass was breached in 2022 by a 3-year-old vulnerability! You would expect that a security vendor is remediating all discovered vulnerabilities swiftly. But that on its own raises a few questions. Questions like ‘Do you really scan all your assets?’ and ‘Do you really track remediation efforts?’. These are questions that every CISO/Security Manager should be asking its vulnerability scanning team. But is it that simple?

Let’s start with the basics. How do you know you are really scanning the entire IT/OT/IOT environment of the company? As most of the devices are relying on the TCP/IP protocol to communicate, somewhere in the company somebody is responsible for maintaining which IP range is used where in the network. This is the starting point for determining if you really are scanning every active IP range for known vulnerabilities. You can track this by implementing this KPI:

If you track this KPI on a weekly interval, you know how accurate the vulnerability scanning is. But how do you know the ‘Active IP ranges’ is accurate? This requires a somewhat data-scientific approach.

Based on the available and centrally collected logging from devices like DHCP servers, Firewalls, IDS/IPS, and other types of network equipment, it is possible to determine with a certain accuracy which IP addresses are used on the network. This list should be compared with the active IP ranges. If unknown IP ranges are discovered, then not only the IP Administration should be updated, but it is equally important to investigate why this has happened.

Even if the KPI ‘Scanned IP ranges’ is 100%, it does not mean all assets are scanned. To bridge this gap, I will introduce three KPIs. ‘Asset scanned’, ‘Asset visibility’, and ‘Asset registration’. The KPI ‘Asset scanned’ is checking that each discovered asset is also checked for known vulnerabilities. The formula for this KPI is:

The KPI ‘Asset visibility’ is checking if all registered assets (the CMDB) are also scanned for known vulnerabilities and/or are visible during the discovery scan. The formula for this KPI is:

The KPI ‘Asset registration’ is ensuring that each discovered asset, is also registered in the CMDB. The formula for this KPI is:

But all these KPIs together don’t provide the insights you need to determine if remediating known discovered vulnerabilities is effective. And here effective means that a discovered known vulnerability is remediated on time. Somewhere in the IT Security policy, there should be a section describing how often a vulnerability and discovery scan should be conducted. For non-internet-facing assets, weekly vulnerability and discovery scanning should be enough in most cases. However, for internet-facing, I do suggest conducting vulnerability and discovery scanning on an hourly basis. Your adversaries are also scanning your internet-facing assets. And at the moment they discover a hole in your defense, they may strike.

Once you have conducted more than one complete vulnerability scan on your environment, you can check if previously discovered known vulnerabilities are remediated on time.

On-time remediation can be determined by checking the number of days between the first time the known vulnerability was detected on an asset and the last time the known vulnerability was detected. If the IT Security policy describes that critical-rated vulnerabilities need to be remediated within 24 hours, then a critical-rated vulnerability is not remediated on time if the number of days is equal to or higher than 1. Based on this logic, the KPI ‘On-time remediation’ can be calculated using the following formula:

If you put all these KPIs in a report and you track them on a weekly base, you can determine if are you really remediating all the discovered vulnerabilities effectively. Especially when you set the goal for each KPI at 99% or higher.

Important: even if you are really remediating all the discovered vulnerabilities on time, you might still be at risk, because this is only addressing the known vulnerabilities. If you are using customized software, you also need to conduct penetration tests on these (web) applications to determine if these (web) applications are safe.