12

When you write a paper about an implementing a classifying computer-vision/machine-learning algorithm, you can simply count the number of positive (correct) negatives/positives, and the number of negative (incorrect) negatives/positives. However is there a way to evaluate an IoT device and its security? How does one go about explaining the evaluation?

The reason I ask this, is because when you write up a paper there is an evaluation section typically, and I'm confused as to how to approach it.

The only thing I can think of is listing the total number of security measures.

Iancovici
  • 223
  • 1
  • 6
  • Iot devices is a very broad classification and will contain the most rudimentary to the most advanced devices. This makes generalizing a device difficult as there will be a varied set of threat models. – Limit Nov 26 '16 at 14:31
  • 4
    CIS usually has benchmarks for measuring the security of various platforms. Here is a link that I found that might interest you: https://www.cisecurity.org/critical-controls/documents/CIS%20Controls%20IoT%20Security%20Companion%20201501015.pdf – Limit Nov 26 '16 at 14:33
  • There is no such thing as there are unlimited unknowns. It's the things that are known that devices are protected against, and it's unknown flaws that get exploited that make a compromised device. There is of course the possibility that a device is not maintained and as new flaws are known the device may not be protected against those. That is something you can measure. There is however no "scale" on getting breached, someone only needs to get in using one way. So if there is one way in or 100 ways in, it's equally vulnerable. Same goes for maintenance; not fixing 1 flaw is enough to fail. – John Keates Nov 26 '16 at 23:42
  • The real question I meant to ask, and I'll edit, is that typically when you write a paper, there's that evaluation section but what would you write in there? – Iancovici Nov 27 '16 at 02:05

4 Answers4

6

When people think of Internet of Things. Most think of various devices with a myriad build of different operating systems and functions.

However it's really not that complex if you look at it as a whole rather than from each individual pieces. Depending on your choice of operating systems. Most have their own security guide and the applications are in fact not much different to our servers and workstations.

Hardening and Security Implementation

You can consider referencing to the following guides but not limited to; for securing your devices.

OWASP IOT Project

IOT Security Foundation

CIS IOT Guide

Windows 10 IOT Guide

Evaluating IOT Security

As for testing I would say starting with this as a guide would help

Owasp IOT testing methodology

Evaluating Software Security

As for the security evaluation of the software, in your case computer-vision/machine-learning. You can additionally adopt evaluation of security on a coding level and configurations rather than on an iot device specifically.

Here's some reference with regards to software security

Owasp Secure Coding Practice

Microsoft Secure Coding Guidelines

scohe001
  • 1,035
  • 2
  • 7
  • 13
Lester T.
  • 1,263
  • 1
  • 9
  • 21
  • Great answer! Version 6 of the Center-for Internet Security (CIS) Top-20 Controls also contains an appendix that provides a Top-20 IoT Controls list – atdre Feb 20 '17 at 00:46
  • yup, the CIS IOT Guide above already mentioned that, Thanks anyway :) – Lester T. Feb 20 '17 at 19:09
  • @ Lester T: Yes, and I upvoted your answer based on that. Can you take a look at my answer here as well and provide some input? Thank you! – atdre Feb 20 '17 at 23:29
4

IoT Security Frameworks

IoT Security Frameworks generally fall into 5 categories: wearable, home, city, environment, and Enterprise. Enterprise software is usually in the purview of OWASP, which has the OWASP IoT project. Enterprise software often transacts PII and payment-card information, which makes it fall under PCI DSS regulations. While not yet strictly regulated, the DHS -- https://www.dhs.gov/securingtheIoT -- the FTC -- https://www.ftc.gov/tips-advice/business-center/guidance/careful-connections-building-security-internet-things -- and ENISA -- https://www.enisa.europa.eu/topics/iot-and-smart-infrastructures -- have also published guidance and supporting documents around IoT Security.

Home IoT devices and apps typically involve iOS (well, or tvOS), Android, or a similar operating system base. If you're an ISV developing apps for these platforms, check out OWASP again, at the Mobile Security Project and also the ASVS standard under the V17 Mobile Security Verification Requirements. If you actively make home IoT devices including a custom OS or stack, then you will also have to involve the regulatory requirements dictated by your location (your country and/or state where the devices will be developed, purchased, and used) as well as what types of transactions and people that will be using the devices. For example, baby monitors might fall under COPPA, heart monitors under HIPAA and HITECH, food and medicine under the FDA, etc.

For the US military, the DIACAP and DITSCAP standards including NIST RMF govern all computing devices, including IoT, especially wearables.

Environment and city-based IoT is much more akin to ICS/SCADA technologies. NIST has selected a program to include environment, city, and ICS to all be under the banner of Cyber-Physical Systems (CPS) and produces standards and frameworks here -- https://www.nist.gov/el/cyber-physical-systems For ICS/SCADA systems, the NIST SP 800-82 Guide to Industrial Control Systems, has been the long standard, but certainly coupled best with the NERC/FERC compliance standard on Critical Infrastructure Protection, especially the sections on System Security Management (CIP-007-5), as well as the ties to International Society of Automation and their all-encompassing ISA/IEC 62443 standard (formerly ISA-99). CIP-007-5 also adheres to other NIST standards on security event monitoring, including NIST SP 800-92 and SP 800-137, but the latest on Continuous Diagnostics and Monitoring comes from the DHS CDM framework. All of these are applicable for Industrial Internet-of Things (IIoT).

IoT Security Platforms

For a platform that can actively scan and produce reports based on IoT/IoE Security frameworks, check out the Pwn Pulse platform from Pwnie Express -- http://m.marketwired.com/press-release/pwnie_express_unveils_industrys_first_internet_of_everything_threat_detection_system-2010032.htm

For other companies working to produce standard interfaces for IoT devices that enable security and reduce cyber-risk, check out (in order of the most-prominent to least-prominent): Bastille, Securithings, Dojo Labs (acquired by Bullguard), and BitDefender (who makes the IoT and smart-things security enabler, BOX). WindRiver, a long-time leader in embedded-system security also released a paper detailing IoT Security -- [PDF] https://www.windriver.com/whitepapers/security-in-the-internet-of-things/wr_security-in-the-internet-of-things.pdf [PDF]

Additionally, only a few companies are bridging the middleware layers between IoT devices and IoT service-layer apps. Certainly the big players are doing their part, but usually with proprietary interfaces such as Cisco Fog Computing (Microsoft Azure IoT Suite, IBM Watson IoT platform, and others have their own ways of doings things as well). The key players changing the game of IoT are working at the all-important instrumentation layer, as well as providing standards for middleware and apps, especially cloud apps. NCC Group published guidelines for these and other security testers here -- https://www.nccgroup.trust/uk/our-research/security-of-things-an-implementers-guide-to-cyber-security-for-internet-of-things-devices-and-beyond/

AWS has published guidance on IoT Security Best Practices -- https://aws.amazon.com/iot/ -- and also provides that middleware layer through their Thing Shadow project (supporting the MQTT IoT protocol standard) -- https://docs.aws.amazon.com/iot/latest/developerguide/thing-shadow-mqtt.html

Splunk has produced a product called the HTTP Event Collector (HEC) to receive cloud-based (Splunk Cloud, AWS, etc) machine data from IoT and future-IoE technologies -- [PDF] https://conf.splunk.com/files/2016/slides/wrangling-your-iot-data-into-splunk.pdf [PDF]. In particular, HEC supports token-authenticated events, as a nice-to-have IoT security feature.

atdre
  • 18,885
  • 6
  • 58
  • 107
0

Q.

Is there a way to evaluate an IoT device and its security?

"When you write a paper, there's that evaluation section but what would you write in there?"

A.

IOT is such a broad area and I doubt there will be a single "catch-all" standard that you will be able to utilise in your documentation. There is also a dependancy on the environment your users will be using your IOT devices in.

Ideally, you will probably want to engage a security specialist and have a vulnerability scan/penetration test on the device. This engagement could also include a code review and discussions around how the device would be secured going forward (patching, vulnerability management etc). The output from this process could then be utilised in your documentation.

You should also provide the security expert with information regarding the intended deployment scenario as this will also have a bearing on the security controls that will be required. The CIS link provided above is a good start and will help you start thinking about the areas that need to be looked at.

nipy
  • 131
  • 2
0

As has been pointed out, "IoT" ("Internet of Things", or more aptly "Internet of Threats") is such as broad classification as to likely be useless. "IoT", in this regard, is little more useful than "electronic device". As such, I very much doubt that you will find some guide to how to approach this type of evaluation in the general case, which would be detailed enough to be applicable in the specific case.

That said, a good starting point for evaluating the security of an Internet of Threats device is likely to assume that its security is fundamentally flawed in some way. (This, by the way, holds approximately equally well for software running on larger systems.) You can then look at best practices actually implemented and try to judge how well the manufacturer is doing in trying to secure the Thing.

For example:

  • Does the manufacturer promise to provide updates for the Thing for some particular amount of time? For how long? What does the owner need to do to update the software that is running on the Thing? Can the owner select between, for example, all updates and only security updates? How are updates validated? What is the manufacturer's track record like?
  • Can the Thing work behind a firewall? Does the manufacturer detail the network traffic initiated and required by the Thing, such that one can reliably configure a firewall to allow desired traffic without opening the firewall to all traffic?
  • Can the Thing function on a network fully isolated from the Internet, possibly with the help of some other Thing installed on premises?
  • Does the Thing use wireless networking, and if so, what standards are supported? Can it join an existing network, or is it limited to networking with others of its kind? How is the network secured, including how does one provide the network credentials? Does this method encourage secure credentials?
  • Does the Thing expose any services to the network? Are the services (including e.g. web administration interfaces) exposed by the Thing protected by strong authentication and confidentiality measures, including HTTPS with at least long-lived self-signed certificates? Can the authentication credentials be changed by the owner independent of the manufacturer?
  • For Things that are designed to interact with other Things (including for example apps on a smartphone, or a manufacturer-provided web interface), look at the network traffic between them while they are interacting. Perform common, simple attacks like man-in-the-middle; are you able to monitor traffic? If HTTPS, do the Things do proper certificate validation or are they happy with a self-signed MD5, 512-bit RSA certificate that you provide?

These are just some starting points, but they are definitely the bare minimum that you should be able to throw at a Thing with little investment in test equipment. A PC running Linux with some common software and two network cards should be plenty enough to get you started with all of the above.

user
  • 7,670
  • 2
  • 30
  • 54