Penetration Testing? A Taxonomy

Initially while I was at Ernst & Young, then through my 7 Elements time, and with the help of many others from vendors and industry have been putting some thought into how penetration testing is currently sold and delivered and how we can improve the process for customers and suppliers. This is a consolidation of posts from other areas, and ideally should be built into the process along with the Penetration Testing Execution Standard.

One of the key issues that we see is that there are different reasons to go broad, or deep. A wide review could aim to identify a range of areas which should be improved, whereas a targeted attack simulation could give good information on what an attacker could do with an opening in the perimeter, combined with weak access controls for example, but may not find many vulnerabilities.

The second issue is with vendors that sell you a "penetration test" but only deliver a lower level of assessment and this can lead to a false sense of security.

So the problem with the "penetration test" term is that most people associate it with this idea that you'll also get coverage of security issues, rather than a focus on specific weaknesses and how they're exploitable.

At the end of the day, an attacker only needs to find one exploitable vulnerability, so while there are certain situations where allowing security testers free reign to go for the crown jewels may be the best option, due to the prevalence of the perimeterised "hard on the outside, soft on the inside" security model, organisations may find a broader approach provides greater assurance for the same budget.

So there is almost a forked model of testing. Typically you would begin with discovery, scanning for common vulnerabilities, and then assessment of those vulnerabilities. After this, the split could be towards Security Assessment (the broad review to find as many vulnerabilities as possible and assess the risk to the business) or towards Penetration Testing (the attempt to exploit and penetrate the organisation to gain access to a particular target).

There will be occasions where these two forks could join up again, where you want a broad review with added information on the extent to which a real world attacker could penetrate.

In order to make it easier to discuss the various stages, our taxonomy is as follows. Please leave comments if you feel improvements are required, and we will develop the taxonomy accordingly:

Discovery

The purpose of this stage is to identify systems within scope and the services in use. It is not intended to discover vulnerabilities, but version detection may highlight deprecated versions of software / firmware and thus indicate potential vulnerabilities.

Vulnerability Scan

Following the discovery stage this looks for known security issues by using automated tools to match conditions with known vulnerabilities. The reported risk level is set automatically by the tool with no manual verification or interpretation by the test vendor. This can be supplemented with credential based scanning that looks to remove some common false positives by using supplied credentials to authenticate with a service (such as local windows accounts).

Vulnerability Assessment

This uses discovery and vulnerability scanning to identify security vulnerabilities and places the findings into the context of the environment under test. An example would be removing common false positives from the report and deciding risk levels that should be applied to each report finding to improve business understanding and context.

Security Assessment

Builds upon Vulnerability Assessment by adding manual verification to confirm exposure, but does not include the exploitation of vulnerabilities to gain further access. Verification could be in the form of authorised access to a system to confirm system settings and involve examining logs, system responses, error messages, codes, etc. A Security Assessment is looking to gain a broad coverage of the systems under test but not the depth of exposure that a specific vulnerability could lead to.

Penetration Test

Penetration testing simulates an attack by a malicious party. Building on the previous stages and involves exploitation of found vulnerabilities to gain further access. Using this approach will result in an understanding of the ability of an attacker to gain access to confidential information, affect data integrity or availability of a service and the respective impact. Each test is approached using a consistent and complete methodology in a way that allows the tester to use their problem solving abilities, the output from a range of tools and their own knowledge of networking and systems to find vulnerabilities that would/ could not be identified by automated tools. This approach looks at the depth of attack as compared to the Security Assessment approach that looks at the broader coverage.

Security Audit

Driven by an Audit / Risk function to look at a specific control or compliance issue. Characterised by a narrow scope, this type of engagement could make use of any of the earlier approaches discussed (vulnerability assessment, security assessment, penetration test).

Security Review

Verification that industry or internal security standards have been applied to system components or product. This is typically completed through gap analysis and utilises build / code reviews or by reviewing design documents and architecture diagrams. This activity does not utilise any of the earlier approaches (Vulnerability Assessment, Security Assessment, Penetration Test, Security Audit)