0

OpenBSD has had much fewer "code execution" vulnerabilities and even fewer exploits than FreeBSD, according to https://cvedetails.com :

However, I wonder if this is an apples-to-apples comparison, given that OpenBSD disables or doesn't support some features that are enabled in FreeBSD by default.

In practice, you'll probably use SMP. You can also use UFS2 instead of ZFS (which is complex) in FreeBSD. You could presumably configure other options to be similar. Would that make FreeBSD as apparently secure as OpenBSD (according to their history of past vulnerabilities and exploits)?

Edit: Emphasized the actual question. People seem to be answering the title, which is shorter, but more likely to be misunderstood.

Edit2: Changed the title to prevent misunderstanding.

MWB
  • 303
  • 2
  • 11
  • 3
    My software doesn't have any CVEs assigned to it, but I can assure you that it's not very secure. – user Nov 18 '21 at 03:35
  • @user Public scrutiny is a factor in CVE stats, obviously. But I would guesstimate that they get a comparable amount of public scrutiny: FBSD is a bit more popular (~3x ?), but OTOH OBSD attracts security-minded folks, so I would guess it's a wash in this department. – MWB Nov 18 '21 at 03:42
  • I'm a little biased, so you'd have to take whatever I say with a grain assault, but OpenBSD has been conducting code audits and they've typically been pretty mindful about security. I would assume that they produce one of the most secure operating systems around (even if they lack in the features department). I've heard that they also have good documentation, so maybe that's also an indication of the codebase. – user Nov 18 '21 at 03:49
  • Who's the audience for this comparison? – securityOrange Nov 18 '21 at 04:19
  • This looks very similar to [Are CVE counts a good indicator of a software's security?](/q/147111/129883) – Fire Quacker Nov 18 '21 at 17:12
  • @FireQuacker In general CVE counts are not a good indicator, because different projects receive different amounts of public scrutiny or are different in nature (kernel vs PDF viewer) But this question is specifically about FBSD and OBSD that **are similar and do get a comparable amount of public scrutiny** (see my comment above). – MWB Nov 18 '21 at 17:52
  • *"People seem to be answering the title, which is shorter, but more likely to be misunderstood."* - actually, the original title *"Can CVE stats be used to compare the security of OpenBSD and FreeBSD?"* asked a very clear question with not much room for ambiguous interpretation. No wonder people answered it. – Steffen Ullrich Nov 18 '21 at 18:41
  • *"Would FreeBSD and OpenBSD have similar known-past-vulnerability stats if they were configured similarly?"* - just similarly? So you would be fine in comparing OpenBSD and FreeBSD both configured with a weak root password, SSH server reachable from outside and also for root? In this case - both would be broken quickly, so yes. Or what exact configuration you would like to compare? – Steffen Ullrich Nov 18 '21 at 18:45
  • @SteffenUllrich *"both configured with a weak root password"* Indeed, the new title can also be intentionally misunderstood (that's why you gotta read the question itself). The title is an attempt at a summary. It's not always perfect. What do you think the title should have been? I'll try to misunderstand yours. – MWB Nov 18 '21 at 20:32
  • @bobcat: Again, what exact configuration would you like to compare? You obviously don't want to use the default configuration but you want to have some kind of similar configuration. The nearest thing to get to this is to define a specific use case, configure each OS to best fit this use case and then compare the achievable security in this use case and the effort which was needed to achieve this security (i.e. comes by default or needs to be explicitly secured). – Steffen Ullrich Nov 18 '21 at 20:41

3 Answers3

2

The original question which is answered here was

Can CVE stats be used to compare the security of OpenBSD and FreeBSD?

Just comparing the CVE is not a reliably way to compare the security of systems. While they are kind of an indicator of the attack surface of a platform, they also indicate how many people actually look at the security of the platform to find bugs and how issues are communicated. Not all issues necessarily get a CVE.

It would be better to look deeper into the details of the OS, i.e. the focus of the development (performance vs. security), the manpower and expertise of the developers regarding security, the kind of issues which might indicate a lack of quality control in critical areas, the design choices for OS and software in terms of robustness even when bugs happen (like heavy use of privilege separation and sandboxing) and more.

... given that OpenBSD disables or doesn't support some features that are enabled in FreeBSD by default.

Even that might be a deliberate design choice, i.e. to proactively limit the attack surface of the platform. It might also be a lack of development resources though. Thus, a closer look why these choices were made is needed too, instead of just counting features.

Steffen Ullrich
  • 184,332
  • 29
  • 363
  • 424
1

CVEs are important but I think the question also refers to flaws, and project's ability to respond to them. Indeed, during any security assessment a company could encounter flaws. Later being referenced as CVEs.

Project's member can easily ignore working on a report of a flaw found by a legit person online (or even during any security audit performed from another company) because they have other work priorities too. (It's like solving a challenge when you have another mission assigned to you.) This applies to companies too. The one I work for evaluate the risk and if it is evaluated being low, the fix is not a priority. When browsing a project's CVE page list you can observe many are considered low or medium risk.

Although both the OpenBSD and FreeBSD projects are not companies per say, they suffer from the same syndrome: Overwhelming. The only difference is "how overwhelmed you are".

Comparing both projects sizes and responsibilities, we can assume FreeBSD's one might be prone to overwhelming. Or at least, easier overwhelmed than OpenBSD. But, if they are not overwhelmed they can allow themselves to fix and patch quickly any flaw reported, no matter if it's a low or very low risk.

Both have different Security policies. Yes OpenBSD is the more secure one could afford... yet. But I don't know if they could handle as much market share as FreeBSD.

Now, let's compare both OpenBSD's and FreeBSD's policies. FreeBSD has a very strict and rigorous security policy involving many intermediate levels from when the flaw is discovered or reported and then disclosed. While OpenBSD find bugs and fixes them. On one hand, OpenBSD security team members are ahead of the flaw. Because they detect any bug and fix it no matter if the bug could be a potential vulnerability or not. On the other hand, FreeBSD security team members put all their effort in fixing and patching, without hesitating contacting (or requesting) external experts to the FreeBSD's project.

Answering your question, I do think CVE stats can be used to compare the security of OpenBSD and FreeBSD. Basing yourself on the CVE stats helps you choose between two products. OpenBSD offers a dedicated security team member fixing bugs one by one and FreeBSD proposes a security team working on fixing flaws. You can then choose if you want a project with fewer people working intensively on preventing flaws or a project with more people working intensively on fixing flaws. Both security policies are good (IMO) and the number of CVE demonstrate FreeBSD's security team capacity to provide response. But the number of CVE for OpenBSD indicates project's security team's workflow is doing great.

To me, comparing both OpenBSD's and FreeBSD's CVE list raises a question: "Which one do you bet on?".

I have seen many people choosing OpenBSD over FreeBSD because "they have fewer CVEs assigned". Without understanding how it's security policy works. Many users ignore why OpenBSD's project has fewer CVEs. But many choose to ignore that FreeBSD can be hardened the way you want.

According you only choose an Operating System based on the security policy. I know, one should not choose any OS just based on one argument but many think the opposite way.

CVEs are the result of a flaw report and a sometimes a disclosed fix, but the flaw can be frozen by a company too. (Leading the flaw to be known to little and some consumers are alerted before others, so the information security professionals from those companies could help fixing it prior any public disclosure)

Ahervi
  • 11
  • 2
1

I didn't stop trying to find answers to my question after posting it, and found a presentation called "Are all BSDs created equally?" by Ilja van Sprundel , who was interested in the same thing:

enter image description here

He audited the code in FreeBSD, OpenBSD and NetBSD over 3 months ("nights and weekends") and found 30, 25 and 60 kernel bugs/vulnerabilities in them, respectively.

Based on this fact, vulnerabilities are still easy to find (in all flavors of BSD). He also argues that the code quality is somewhat better in OpenBSD, in addition to there being significantly less code/functionality in it: 3MLOC in OpenBSD vs 7MLOC in NetBSD and 9MLOC in FreeBSD.

His presentation: https://media.ccc.de/v/34c3-8968-are_all_bsds_created_equally

MWB
  • 303
  • 2
  • 11