How to discover known vulnerabilities (CVEs) in Go executables?

Question

Tools that discover CVEs need to create software bill-of-material. In many cases it can be done simply by inspecting files on the target system e.g. RPM or NPM index.

Would it be possible to do composition analysis on Go executables in contrast to Go source code?

Is there any tools attempting to do that?

edit: the question is about executables programmed in go language in general and how to automatically discover transitive dependencies of those executables as part of executing vulnerability scan for a target system.

The only tool that I can think of is something like https://www.cvedetails.com/vulnerability-list/vendor_id-14185/product_id-29205/Golang-GO.html which lists out known vulnerabilities in accordance to the Go product, but what other types of information are you hoping to glean from Go executables and why would it be difference to vulnerabilities to Go source code? — NASAhorse, Sep 27 '18 at 07:56
Some vulnerability scanners work by indexing packages without requiring access to source code or without advance knowledge of what software is running on a target - they might be e.g. indexing Docker container or connecting directly to target system via SSH and create software bill-of-material automatically. But so far I've not found one that would support indexing executables programmed in go. — tsaarni, Sep 27 '18 at 15:34
Possible duplicate of [How to find out what vulnerabilities X product has had/has?](https://security.stackexchange.com/questions/185923/how-to-find-out-what-vulnerabilities-x-product-has-had-has) — , Sep 28 '18 at 14:49
The vulnerability scanners I've used cannot discover CVEs in go executables. The difficulty is how to automatically produce the bill-of-material. As an example: Kubernetes source code repository contains over 200 LICENSE files, which could be considered as an indication of the number of projects it depends on. Optimally all of those should be part of CVE search. — tsaarni, Sep 29 '18 at 07:16

score 2 · Answer 1 · answered Oct 01 '18 at 04:46

Not really. I mean, in theory you could, but it's absurdly unlikely that anyone will.

Go is a relatively safe language, so vulns in Go code tend to be logic errors, not mechanical ones. This means that vulnerabilities are marked not by patterns in the binary, but attributes of the control flow.

Essentially for detecting a given CVE you'd have to construct the opcode sequence that corresponds to the vulnerable logic, and match the binary against that pattern. This might be simple for a trivial pattern, but trivial patterns are unlikely, and it's a lot of work for essentially no benefit.

After all, vulnerability detection is fundamentally a supply chain problem. If your upstream can't source vulnerability metadata, then your whole trust model is borked. If you are the upstream (you build the binaries) then you need to start embedding metadata in your package details (eg. source code provenance) that can be used to determine CVE relevance.

I agree, detecting vulnerable logic is not easy and I'm after less ambitious goal: simply listing packages that contribute to a build of statically linked executable, then comparing the list to CVEs. You bring very interesting proposal about embedding metadata in the package details during build. Go compiler itself could do this in order to facilitate scanners that look for CVE matches. I wonder, is there related efforts already? — tsaarni, Oct 02 '18 at 14:11

slayer · Answer 2 · 2019-07-02T02:30:15.627

I understand that you want some vulnerability scanning/auditing that is capable to find vulnerabilities on Go executables by inspecting it. I already read some articles that fit on keywords such as "golang vulnerable function" expecting to encountering some lack of security functions alike in C language that fight against stack overflow (i.g., strcpy, gets, sprintf, ...), memory adjacent overflow (i.g., strncpy, strncat, ...) and on and on. However, I haven't found ways to discover flaws this way, but as you aforementioned on several CVE listed in CVEDetails, these known vulnerabilities are possible to detect using NIST National Vulnerability Database (usually used into an unauthenticated vulnerability scanning, that's why some vulnerability scanning can found some flaws without requires credentials - i.e., Nmap and Nessus) and you can also use the vulnerability auditing method such thing like Open Vulnerability Assessment Language (aka OVAL Language - An open and publicly available security content, and to standardize the transfer of this information across the entire spectrum of security tools and services.¹ -, usually used in authenticated scanning [i.e., OpenSCAP]), namely, you can read about that and create your own "rules" to detect those known vulnerabilities.

Publicly information security vulnerabilities

There is an information vacuum in some of publicly known vulnerabilities database (i.e., NVDCVE and CVE) hence it is fairly biased to false negative (where there is insufficient information for scanning to be able to trigger the presence of vulnerability) and, rarely, it can also happen false positive alerts (which means that can report vulnerabilities that do not exist on the environment.) If you want to perform a vulnerability assessment upon the package information on the machine that you have access to, I encourage you to choose the OVAL Language as your partner on this journey.

Vulnerability Scanning and Vulnerability Auditing

Both are the focus on discovering system weaknesses, although you can't compare them, as each is different purposes. The vulnerability auditing (i.e., Lynis and SCAP) performs tests to determine how the system is well configured looking at in every corner as possible. The other method is the vulnerability scanning (i.e., Nessus and QualysGuard) that is performed to discover software flaws, detecting some running service or installed application, however, consequently, does several tests against them and reports their Result

Analysing a known vulnerability in GoLang

For example CVE-2018-6574: https://www.cvedetails.com/cve/CVE-2018-6574/.

If you go through "Scroll to -> OVAL Definitions" you'll see the "OVAL Definition ID" (oval:com.redhat.rhsa:def:20180878), click it and scroll down to see a conditional structure that assesses the current and related vulnerability. If you want to know how to write a signature that makes vulnerability assessment on certain CVE, I recommend you read about Open Vulnerability Assessment Language (OVAL Language)², mainly the OVAL Content Creation Tutorial³ article.

Despite all of these vulnerabilities listed on CVEDetails, that's not means that are publicly OVAL Definition for all of them, for instance, CVE-2018-7187⁴. This vulnerability is in GoLang, version 1.9.4:

The "go get" implementation in Go 1.9.4, when the -insecure command-line option is used, does not validate the import path (get/vcs.go only checks for "://" anywhere in the string), which allows remote attackers to execute arbitrary OS commands via a crafted web site.

Publish Date : 2018-02-16 Last Update Date : 2018-03-13

Refer: https://github.com/golang/go/issues/23867

According to the [CVE-2018-7187] CVEDetails information that I mentioned above, the operating system is Debian GNU/Linux 7, albeit you can validate it into whatsoever Linux-like operating system with. Whether you putting all information together that composes an OVAL signature:

OVAL Definitions: Means what you'll be checking for. The "OVAL Definitions" is the category that can be composed with multiples "Definition" class as well as the other categories described below;
- Criteria: This is a class that is associated with the OVAL Definitions category. The role of this class is a forward routine that corresponds to what you are currently assessing to the "Test";
OVAL Objects: This specifies the raw information that you expects to found. Usually, this topic contains the literal names (i.g., application, windows registry key, etc)
OVAL Test: This one performs a logical test to what is checking. Roughly, this guarantees at least the existence of n an item to trigger as true.
OVAL State: This specifies the information expected to be compared to the information collected.
Result: When the information expected (from OVAL State) is evaluated from the what is collected, the matchup value (being true or false) is stored to the OVAL Results as a report of the outcome assessment.

Note: Read the refer 3 for more information.

Detecting CVE-2018-7187 using OVAL Language:

You can make the vulnerability assessment depending on your target. For Windows, the OVALid⁵ can be as a stopgap, and on another hand, for Linux-like OS you can use OpenSCAP Base⁶.

In the following steps, I will describe a little about how you can build your own lab to make vulnerability assessment on any software applications into a Linux-like OS environment.

1. Create the lab:

The required package are:

autoconf automake libtool make libdbus-1-dev libdbus-glib-1-dev libcurl4-openssl-dev libgcrypt20-dev libselinux1-dev libxslt1-dev libgconf2-dev libacl1-dev libblkid-dev libcap-dev libxml2-dev libldap2-dev libpcre3-dev python-dev swig libxml-parser-perl libxml-xpath-perl libperl5.22 python-dev libbz2-dev librpm-dev swig

1.2. Compiling OpenSCAP

$ git clone https://github.com/OpenSCAP/openscap.git
$ cd openscap
$ ./autogen.sh && ./configure && make

Note: If you have some unpredictable installing error read this OpenSCAP Guide, Developer's operations⁷. If you came across with linking error because not resolve "rpmlogClose()" symbol, go to src/OVAL/probes/Makefile and edit the line 1210 adding -lrpmio.⁸

2. Prepare the OVAL Language rule:

I have created the rule to detect CVE-2018-7187, tested on Ubuntu 18.04.1 LTS: https://gist.github.com/slayerlab/b2a358f13ab267f2e9543bb9f9320ffc

3. Use OpenSCAP to perform the vulnerability assessment:

On local machine:

$ oscap oval eval linux-definitions-vulnerability-oval.xml 
Definition oval:com.stackexchange.security:def:666: true
Evaluation done.

On remote machine you can use oscap-ssh with the follwing commandline:

$ oscap <user@domain> oval eval <oval_rule>

or use oscap-ssh, a bash script by mpreisler⁹:

$ bash oscap-ssh slayer@0.0.0.0 22 oval eval ~/PoC/linux-definitions-vulnerability-oval.xml 
Connecting to 'slayer@0.0.0.0' on port '22'...
slayer@0.0.0.0's password: 
Connected!
Copying input file '/home/slayer/PoC/linux-definitions-vulnerability-oval.xml' to remote working directory '/tmp/tmp.bh28nES8c9'...
linux-definitions-vulnerability-oval.xml                                                                  100% 3386     3.0MB/s   00:00    
Starting the evaluation...
Definition oval:com.stackexchange.security:def:666: true
Evaluation done.
oscap exit code: 0
Copying back requested files...
Removing remote temporary directory...
Disconnecting ssh and removing master ssh socket directory...

Thanks for extremely good intro to OVAL and OpenSCAP! In the example, RPM is used to discover the list of packages. The original problem still remains: how to discover list of packages (transitive dependencies) that were used to compile a Go executable — tsaarni, Oct 02 '18 at 13:55
@tsaarni Each XML node presence inside of *root node* (`oval_definitions`), in OVAL Language, are refers as *probes*. Regardless of the package manager, either `dpkg` or `rpm` and even more. The probe which is the charge to perform the *test* using RPM package is `rpminfo_test`. You can see all available probes in OVAL Language version 5.10 on the "test listing" document: `https://oval.mitre.org/language/version5.10/test_listing.html`. — slayer, Oct 02 '18 at 19:19