How are my scores calculated?

Customers who use Meterian to discover and fix vulnerabilities often ask this question.

Meterian’s assessment report displays two scores labelled Security and Stability, which were described in an earlier post.  Assuming a project has no vulnerabilities until one is found, Meterian’s Continuous Security Platform initially assigns a project a score of 100 until it finds a vulnerability.   We trust you understand why no code is ever 100% secure and free from vulnerabilities.

Finding a vulnerability will result in points being deducted from the initial score “perfect” score of 100 depending on the vulnerability advice type and its severity.  

We have three vulnerability advice types:

  • Security  This type of advice is usually related to a very well known security flaw in a library, such as the infamous CVE-2017-5638 that caused the recent Equifax disaster. Deductions for the Security score are based on the Common Vulnerability Scoring System (CVSS) score of the vulnerability advisory and then multiplied by a factor of 5.  In this case, CVE-2017-5638 was assigned a score of 6 by CVSS and therefore Meterian would deduct 6 x 5 = 30 points.
  • Defect  This type of advice is used to report a serious defect in a library that can potentially affect the stability of your system. For example, the memory leak that affects certain versions of Hibernate is a very well known ORM library defect. Deductions for defect types depend on their severity:
    1. minus 20 for each Defect advice with high severity
    2. minus 10 for each Defect advice with medium severity
    3. minus 4 for each Defect advice with low severity
    4. minus 1 for any direct depending library where a patch version is available
  • Suggestion  This type of advice provides a suggestion from Meterian for projects to swap a library for another or to upgrade to the next version for better code maintenance which may impact the security or stability of your code.

The amount of points deducted depends on the severity of the vulnerability.  In our view, lower scores indicate higher risk from vulnerabilities detected in your code. We prefer to err on the side of caution and indicate a strong signal for security risk rather than send a mild signal that might result in security risks being overlooked.  

Some may think our scoring applies a harsh judgement.  Recalling our previous example of CVE-2017-5638, this security advice type has a high score of 10 and therefore Meterian would penalize the Security score of projects with this dependency by -50 points.  Isn’t it better to receive a stronger alert to call attention to bolster your code’s security rather than risk a more mild alert that could lead to undeserved complacency?

So please don’t be intimidated by seeing Security or Stability scores of 0!  Make haste to fix the problems reported as soon as possible to avoid software decay and security risks that are preventable.

 

How are my scores calculated?

THE REPORT

In the previous post we introduced our badges that give an immediate feedback on the status of your code. Badges are handy, but we are really trying to give developers the best feedback possible to allow them to fix possible defects and vulnerabilities quickly.

For those reasons we have created the Report

report

The Report gives you a detailed analysis of your dependencies, listing all the libraries with known security vulnerabilities as well as the ones who have received bug fixes and should be updated. It’s really simple at this point to decide what to fix, and see the Stability and Security indicators increase in the next Meterian scan (see also our FAQ: How frequently does Meterian scan my project?)

In the top part of the page you can notice a selector to visualise the report for each of your project branches; just remember that creating a report takes a few minutes, so the first time you switch to a new branch allow for up to 5 minutes for the report to appear.

Interested in learning more about Meterian? We are looking for companies willing to trial it and work with us to build the next-generation Continuous Security platform. Contact us at info@meterian.com for more details.

THE REPORT

Security and Stability

As we are trying to bring Continuous Security into every software project, we came up with two simple indicators to quantify the health status of a codebase: we called them Security and Stability.

Security measures how likely is a codebase to be affected by security vulnerabilities. A value of 0 stands for “very likely to be insecure”, while a value of 100 is, of course, very secure according to our analysis.  Several factors can decrease the security score: depending on a library with known security issues is one of the major, as well as displaying one of the common “mistake patterns” in the code. We’ll certainly talk more about that in future posts.

The Stability indicator shows how likely is code to be subject to critical defects. While not directly related to software security, critical defects can cause the application to misbehave, crash, or perform poorly. Similarly to the Security indicator, the Stability indicator is calculated using a mix of static code analysis and assessment of the libraries in use.

When you integrate the Meterian Continuous Security platform in your application, it is a good idea to display the two badges in your project page (or in the README file if you use one of the popular platforms like Github), to immediately have a clear indicator of the health status of your codebase. Of course it is also possible to display a full report of all the issues detected by the latest security scan – that will be the topic of the next blog post.

Security and Stability