Automated Vulnerability Alerts for Embedded Linux

Embedded Linux is all around us. It’s in our routers, IoT devices, and medical devices. Past events, however, have shown us that the Internet of Things (IoT) gives us just as many new vulnerabilities as new opportunities.  

‍

Embedded IoT devices are a more serious security risk than typical consumer computers because of their slow update cycles. Some don’t allow remote upgrade. This means that if a vulnerability is found, there is no cost-effective way for the manufacturer to update the device with a patch. Even if the manufacturer can remotely update it, they may not be keeping up to date with the latest security vulnerabilities or updates.

‍

Regulatory agencies are taking notice. The FDA’s premarket and postmarket cybersecurity guidances make it very clear that they are concerned with medical device cybersecurity. It’s up to medical device manufacturers to be on top of the problem through joining ISAOs, setting up coordinated disclosure programs, and issuing security updates.

‍

In this post, we’re going to show you how to run automated checks of third-party packages for known vulnerabilities (via the National Vulnerability Database) and how to integrate that into your Continuous Integration System (CI). This is just one part of monitoring your device’s security effectively.  However, this process is easy to do. If you’re building an embedded device, there is no excuse for not having something like this in place.

‍

Tools Needed:

• Continuous Integration Framework: We use Jenkins for projects at Promenade Software, but any CI will do.
• Build and Version Management Framework:  Examples include Yocto, RPM, or a SWID generator. We use Yocto for our embedded linux devices. Your package manager or SWID tool should be able to generate a list of all installed packages and their dependencies. (You do know every package and its version number that’s installed on your device, including dependencies… right?)
• Vulnerability Analysis Tool: We built a tool called CVE Checker that compares your list of packages and versions to the list of Common Vulnerabilities and Exposures (CVEs) in the National Vulnerability Database. We’re releasing it under the GPL license so that everyone can benefit. You can clone it here (Vulnerability Analysis tool).
  (Side Note: If you’re building a web application check out the OWASP Dependency-Check Jenkins plugin)

‍

The following steps should be performed by the CI after every build of the system, preferably right after your unit tests.

Step 1) Get the list of packages (including all dependencies) on your device.

• Using yocto: See the manual for enabling build history. The package list is in a file called installed-packages.txt
• Using rpm: You can get the list by running `rpm -qa`.
• Other: You need to generate a list of  ISO/IEC 19770-2:2009 compatible SWID tags. Choose a tool that matches your operating system and workflow best.

Step 2) Run the CVE checker on that list.

• Clone the checker from github.
• Run download_xml.sh to download the latest NVD from NIST. Do this the first time and then at regular intervals to keep up to date.
• Run cli.py on the package version list.  If you need help formatting the command, you can view the help with the “-h” flag.  example: ~/tools/LibScanner/cli.py --format yocto -i "ignored_cves.txt" "installed-packages.txt" ~/tools/LibScanner/dbs/ > cve_test.xml

The output from this step is a JUnit-style XML document. Run the output from the tool through the same reporting mechanism used by the unit tests. Address any failures through updates, patches, or other mitigations as necessary.  If a CVE has been sufficiently mitigated through alternative means and is still causing the tests to fail, you can add it to the "ignore" list using the -i option.

‍

The ignore list is a simple text file with a comma-delimited list of CVEs to ignore and why (e.g. mitigating control taken, false positive, etc.). All tests that are in the ignore lists will show up as “skipped” in the report.

‍

Example :

CVE-2016-9318 , Device shall not allow arbitrary xml parsing under any circumstance
CVE-2016-6354 , False Positive - Flex not actually on device

‍

Have your CI run this build every time someone checks in a change, and at a regular interval (e.g. once a month) after the device is in production. That way, you’ll be alerted by a failing unit test if there are any new vulnerabilities added to the NVD in the future. Every failure needs a mitigating control, regardless of how small.

‍

In addition to Promenade's other post-market cybersecurity services, we run this procedure every month on every embedded-linux device that we design. This helps us keep up to date on new vulnerabilities that impact our clients' devices and respond faster and more effectively.