12

Looking to create a form where developers can submit requests for packages to be installed. We want to create a list of questions that can help us determine whether or not a package is safe. What are some important questions to include in the form for our developers?

My list so far:

  1. Package Type: NPM, PYPI, etc...
  2. Package Name:
  3. Package Version:
  4. Package Release Date:
  5. Explain Use Case of the package:
  6. Provide the Package Documentation
  7. Commit history? Actively maintained and updated? How many people can make commit changes? Are changes automatically approved or are they reviewed?
  8. Are there open Bug Reports? How many? How long have they been open?
  9. Any active or previous vulnerabilities listed in NVD? https://nvd.nist.gov/vuln/search?results_type=overview&query=Cloudinary&search_type=all&form_type=Basic&isCpeNameSearch=false
  10. What dependencies does this package require.
schroeder
  • 123,438
  • 55
  • 284
  • 319
user277711
  • 123
  • 1
  • 4
  • 9
    How would any of that list be helpful for determining if a package is safe except for #7-9? Would you determine it is safe based on the form questions? Why not, you know, test the package? – schroeder May 04 '22 at 13:47
  • 6
    Posting as a comment because I cannot construct this into a complete answer - dont make the engineers fill out a form. Automate this as part of your CI/CD pipeline. In Java with Maven one can use tools like sonarqube and dependency-check to show, list, and find CVEs on public packages. JFrog and Snyk offer commercial solutions. Using the output from these automated tools to then SPECIFICALLY follow up with engineers about risky libraries or unused libraries will be far more productive. You can even pre-fill probably 2/3 of the form with some simple automation. – Freiheit May 06 '22 at 17:03

3 Answers3

43

I highly doubt that a process to request approvals for new third-party packages will have the desired effects. I've worked for organizations that have tried to introduce similar processes, and they tend to fail. The approval process rarely fits into the speed and cadence of development, leading to problems like teams not being able to execute on their planned work or bypassing the review process entirely and dropping key aspects of third-party package review and selection.

Especially in agile organizations, when the need or possibility for pulling in a third-party package as a solution arises, the team usually doesn't have a lot of time to make a decision. The work is already in progress and they need a rapid decision to begin to move forward to design, build, and integrate solutions.

The first step is to give the team the knowledge needed to select appropriate packages, considering things like license terms and the overall health of the different options. The health of the package may consider any number of factors, but some that I've seen are things like how responsive the developer is to questions/issues in official support channels, how active the user community is (including third-party channels like Stack Overflow or various forums), the number of open issues and/or time to resolve defects, number of open pull requests, age of pull request, number of committers and who the committers are, frequency of commits, frequency of releases, number of times the package is a dependency, number of downloads (per unit of time, in some cases), number of dependencies, and documentation (readme, contributor documents, funding information).

Unfortunately, no one but you can determine what factors are most important. A big factor are the risks associated with the system that you are developing, along with the tolerance for risk for the developing organization as well as the users and customers. Some contexts are very sensitive to risks, while others are very tolerant.

Snyk and Synopsys have tools that track common open source components and make some health information public. Their ratings and criteria may not be totally appropriate for your organization and you may need to add guidance on how to interpret their data or what to do when components are not in their databases, but this may give you a good starting point to make things easier for the teams looking to include open source components.

Giving the developers doing the work the training and the tools needed to compare options and make informed decisions based on guidelines is important. Taking these tactical decisions away from the team will only slow down the development effort and leave the teams unempowered to make the best design decisions.

Once a package is incorporated, there's also ongoing maintenance. The use of software composition analysis tools can scan your software, find dependencies, and monitor those dependencies. You can be alerted to things like new versions, new vulnerabilities, or packages that no longer appear to be maintained. When these alerts come through, the development team can triage them to apply patches (or other mitigations), defer patches for a later time (if the vulnerability is low risk or there are other mitigations already in place), or identify when it may be time to migrate away from one dependency to another solution.

Depending on your threat model, you may also need to consider other ways to mitigate risks. Even with the appropriate reviews, there are cases of developers yanking their packages from the Internet, purposefully injecting malicious code in new versions, or not adhering to standard versioning schemes and breaking dependent systems. Versioning pinning, standing up mirrors for your dependencies, or building your dependencies from source may mitigate further risks. For open source dependencies, you may also be able to scan the source with your internal vulnerability scanners to further mitigate risks of malicious code.

Thomas Owens
  • 1,022
  • 8
  • 9
  • Excellent answer! I would like to add (based on (semi-)failed attempts to impose strict analysis on external dependencies: JFrog Xray. Seriously overpriced, but can do binary analysis and has a database updated dynamically almost twice a day. I believe it can also handle licensing issues (for composite packages like Docker Images). – GCon May 05 '22 at 11:50
  • "I highly doubt that a process to request approvals for new third-party packages will have the desired effects." I disagree. The proposed process seems to put all the burden for vetting 3rd party packages on the devs that request them, with the security engineer just having a quick look and approving. As long as the security engineer checks the requests regularly, the devs who need the package can easily plan for it. And a form has the advantage that the vetting process gets documented. – Joooeey May 06 '22 at 15:11
  • 1
    @Joooeey Did you read the rest of the paragraph? I present two problems - significantly slowed development processes or developers entirely bypassing the process and not doing any security review of new packages. It also can slow down the introduction of patches if new patches need to go through this process. I've yet to see a form-based approval process done by a security organization actually add to the security of a software system. – Thomas Owens May 06 '22 at 15:14
  • I did. My point is a form with straight-forward checks can be pretty fast if well done. What's a faster alternative though? You propose continuous checking of the code base but then what are you gonna do if you find a package considered a security risk? Sure it takes a lot more time to replace a package that's already in use. – Joooeey May 06 '22 at 15:19
  • 1
    @Joooeey When a package in the codebase is flagged as vulnerable, you triage it. Depending on how the package is used, the vulnerability may not be exploitable or there may be compensating controls that mitigate the exploitability. Prioritize patching or placing mitigating controls among all of the other work before the development teams. If you're keeping your packages up-to-date, it should be very easy to go up a patch with minimal impact. If you have good automated test coverage, that also reduces the risk significantly. – Thomas Owens May 06 '22 at 15:23
  • "Once a package is incorporated, there's also ongoing maintenance. The use of software composition analysis tools can scan your software, find dependencies, and monitor those dependencies. You can be alerted to things like new versions, new vulnerabilities, or packages that no longer appear to be maintained." I think you buried the lead on page 3. Imagine if you approved Log4J and never looked back. – JimmyJames May 06 '22 at 20:44
  • 2
    @JimmyJames Yeah. I struggled with that a bit when writing the answer. But I opted to walk through the process from selection through maintenance. – Thomas Owens May 06 '22 at 22:13
13

First of all, be aware that there is no guaranteed security. For example, the npm package node-ipc decided to wipe the hard drives of Russian users for political reasons. faker.js decided to just stop existing and thus breaking a lot of code. Developers and maintainers are people too, and these people sometimes act irrationally, sometimes maliciously. In essence, you cannot judge what people will do in the future. You cannot protect against a developer acting maliciously. But you can make judgements about the "health" of a project.

What does "project health" mean?

Now that we have established that we cannot protect ourselves from malicious developers, we have to look at what we can protect ourselves from: Developers losing interest.

If you've worked in a large company, you've likely seen codebases depend on some library that was written 2003 and last updated 2004, when the developer just vanished. The library is never replaced, because "it's too deeply integrated into the product and we would have to re-write everything". The library has several known vulnerabilities, averaging at a CVSS score of 7.6, but again, it's not replaced, because there is no replacement without lots of work.

This is the precise scenario you want to avoid. And the likelihood of that occurring or not occurring is determined by "project health".

Positive factors contributing to project health are:

  • Large company backing
  • Long development history
  • Concrete end-of-life plans
  • Many developers and maintainers

All of these make it less likely that a project will suddenly stop receiving updates and leave you stranded. So the healthier a project, the more likely it is that you will receive updates, including security updates, for the lifespan of your product.

Do you even need the dependency?

The first question you should ask yourself is is the cost of developing this in-house greater than the risk of taking a package? For packages like left-pad, the answer is clearly "No" and developers should write it themselves, rather than taking on yet another dependency. For other code, for example, cryptographic code, you're likely better off using a well-vetted package than developing your own.

Remember, coding it yourself means you are in control of the codebase's health.

How healthy is the project?

As I mentioned above, there are several factors contributing to a project's health. Note that these don't include things like "number of vulnerabilities", but more the project team's ability to fix those vulnerabilities.

If a project or package doesn't seem actively maintained, then you're better off either finding an alternative or developing it yourself. Which of these alternatives seems better to you has to be judged on a case-by-case basis.

What about project dependencies?

Of course, if those projects have dependencies (and they likely do), then you will have to vet those as well, using the same process. It may very well be that your shiny new super-awesome.js project depends on 300 other things in total, and you're likely better off not using it.

In Short

  • You cannot protect against malicious developers
  • You can make reasonable estimates about reliability
  • Don't take a dependency if you don't have to
  • Favor mature projects over small projects
  • Avoid nested dependencies when possible
  • 2
    In case someone is wondering why `left-pad` was mentioned: [How one programmer broke the internet by deleting a tiny piece of code](https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/) – Andrew T. May 05 '22 at 07:35
  • 1
    Why can't you protect against malicious developers? All the examples you have mentioned could have been prevented if the users of the packages just set a fixed version instead of taking the latest one. – Aviv Aviv May 06 '22 at 16:03
  • @AvivAviv Say my package targets `2.6.1` specifically, because at the time of this writing, that is the latest stable version. 7 months pass, and my patch management tool suggests that a vulnerability has been found in `<=2.8.0` and l should update to the latest stable. The latest stable, however, includes a malicious commit by the developer. –  May 06 '22 at 19:32
  • 1
    @MechMK1 if you set a fixed version just to update it without investigating that the new version is OK then might as well take the latest. Of course, the resource of the security team of, say, an enterprise business are limited and they can't conduct a code review of every single open source they want to update, however, basic research can be conducted. for example, the node-ipc protestware was discovered pretty quickly. – Aviv Aviv May 07 '22 at 09:18
  • @AvivAviv What about malicious backdoors that *aren't* discovered very quickly? –  May 07 '22 at 11:30
  • 1
    @MechMK1 if some open-source developer is playing the malicious attack for the long run, not activating the payload right away, then they risk the payload being discovered before they can execute their plan. Of course, it is still technically possible to inject a hidden backdoor for a long time in a widely used library, however, it is very unlikely no one will notice, including another contributor (if any) take for example the PHP interpreter backdoor from a few months ago. But to say that you can't at least somewhat protect against malicious developers? – Aviv Aviv May 07 '22 at 21:35
0

From a well-behaving company's perspective, the most important thing is the LICENSE that the software package has been released under.

The technical quality or risks of the package may hard to judge, but the legal situation for packages is very clear.

I've worked for many companies that have a complete ban on [L]GPL-3, AGPL, and similar strong "copyleft" licenses because of the implications that they have on the company's proprietary code in a product.

Roger Lucas
  • 161
  • 3
  • 2
    I disagree on LGPL specifically, as it was designed to allow usage in proprietary code without releasing that code under the LGPL or similar license. –  May 06 '22 at 19:31
  • @MechMK1 you're right about the code disclosure, but GPLv3 grants the rights on User Products to replace the GPLv3 code by the user... so it means that signed images, etc. aren't really possible. LGPLv3 doesn't remove this right. So, for any product that has a signed image, GPLv3 and LGPLv3 are a problem. – Roger Lucas May 07 '22 at 14:15