5

I did some research on the topic (as I'm new to it) and it frankly seems too good to be true. A company called Darktrace for instance, makes use of AI algorithms and ML to autonomously detect threats within the environment.

Say I have an infected network. It's been infected for as long as I can tell. I then run Darktrace on this infected environment. Would it take the infected network as the "normal" network? I can see how it would work if it was just an isolated desktop, it would just compare the activity to other desktops in the network. But how would it be able to tell the whole network is infected ?

schroeder
  • 123,438
  • 55
  • 284
  • 319
Marco Neves
  • 53
  • 4
  • Please restrict yourself to one and narrow question per post. While the question about Darktrace would fit this description the second question is kind of broad. Apart from that the second question is not tightly coupled to the first, i.e. it could be asked independently from the first. – Steffen Ullrich Oct 09 '17 at 06:31
  • 1
    The title of your question still broadly asks about ML in general while your question itself is now focused on the specific product Darktrace. Please align title and body of the question. – Steffen Ullrich Oct 09 '17 at 07:26
  • Is ML machine learning? If it is then its used as a buzz word in this area and not as something that is actually useful. – daniel Oct 09 '17 at 08:49
  • 1
    @MarcoNeves: It is unclear what you are asking. Do you want to know about drawbacks of ML in general as the title suggests or do you want to know how a specific product behaves in your specific setup as the body suggests. Note that other products might behave differently in this setup i.e. there is no general "ML behaves like this" and most products are also not pure ML either. – Steffen Ullrich Oct 09 '17 at 08:52
  • @daniel not a buzz word, it is actually useful, people assume that it is magic, though – schroeder Oct 09 '17 at 08:55
  • @schroeder wikipedia states Machine learning is a buzz word. This is like people using fuzzy logic where a PID controller would work just as well. – daniel Oct 09 '17 at 08:57
  • 2
    @daniel well, if wiki says it .... Except it was coined in the 1950's by IBM and has been a mainstay in computer science and mathematics for decades, but yeah, let's go with a marketing report from less than a year ago without any context ... – schroeder Oct 09 '17 at 09:02
  • ***To all voters:*** It looks to me that the OP chose an example of an ML implementation, but it not asking specifically about that vendor's product. – schroeder Oct 09 '17 at 09:30

1 Answers1

9

You are asking a couple different questions at once: how is ML implemented by a certain vendor, and how could it be implemented to be effective. Let's focus on the later.

I have designed such algorithms for a major global financial institution, and I can give you the broad strokes.

It's not too good to be true. Any security analyst will look at the logs, get to know how people operate in the company, and "get a sense" for what "normal" looks like. It's called 'baselining'. After some time, you "just know" when something looks fishy and you need to investigate because:

  • The network goes quiet every Friday afternoon (for reasons that are obvious to any human in the office at the time)
  • Network bandwidth gets maxed when the World Cup is on (streaming, not DDoS)
  • That server always throws that error (a bad config no one knows how to fix)
  • The user can never remember their passwords and always gets locked out
  • We just turn a blind eye to that exec because everyone is afraid to tell them to behave more securely
  • etc.

Every other oddity, whether excessive errors or abnormally low errors (hey, why didn't that server throw that error?) deserves investigation. This used to be infosec bread and butter, but now goes under the fancy term of "threat hunting".

But this baselining is easy to teach a computer. And you are correct, it all depends on the ability for the computer to understand what is 'normal'.

It would be a crude approach to take the current state of the network and determine that that is 'normal'. You would train your algorithm to accept bad behaviour. You still do this as a factor, but you cannot depend on it entirely. You need another way of looking at the data. There are a couple ways to add another perspective.

There are lots of fancy terms, but when I talk to non-algorithm people, I use the terms: 'weirdness' and 'badness'. Baselining the current state helps to determine 'weirdness'. If the network is already hacked, and the hacker is freely floating around the network, then that is not 'weird' for this network. It's 'bad', but not 'weird'. If we are good at determining 'weirdness', then we can see if a new hacker comes in, or if the current hacker changes tactics.

We can augment the network baseline by determining what is 'weird' for user types by baselining subsets of the network. Devs act a certain way, execs act a certain way, cleaning staff act a certain way. If a cleaning staff account normally acts like an executive account in our current state, then we have something to investigate, even though it is 'normal' in the current state. So, this recursive baselining is one way to augment the ML perspective.

Another way to add to the perspective is to define 'badness'. Things might be normal, but we can define that activity as categorically 'bad'. It might be 'normal' for this printer to be trying to log into every server in the DC, but we know that this is categorically 'bad'. So, if we can input these parameters into the algorithm (a lookup into a signatures table), we can expose the badness in our otherwise normal network activity.

I first got into UBA after I ran a demo of a new UBA product about 3 years ago. I knew my network backwards and forwards, and instinctively knew what was normal and what wasn't. I looked into the UBA product as a backup for me and my small team to cover that skillset for when people were on holidays or if people left the team. I wasn't expecting much. But the product included this 'badness' perspective that instantly (within 2 hours) exposed badness in my network that I never knew was there even though I had baselined and was performing threat hunting daily.

ML is not too good to be true. It's not perfect, but it is better than humans at breadth, speed, and consistency.

schroeder
  • 123,438
  • 55
  • 284
  • 319
  • Very insightful. Though that last sentence had me wondering what makes it not perfect. – Marco Neves Oct 09 '17 at 09:11
  • Many critics will say that if it cannot detect 100% of valid anomalies, then it is useless. And no ML process can detect 100%. There will always be false positives, and false negatives. People's behaviours are too difficult to sum up with an algorithm and changes occur slowly as the culture changes, and behaviours can suddenly change without warning. One standard example is a fire alarm. UBA ML algorithms freak out when this happens because everyone suddenly acts 'weird'. The UBA operators then have to wade through all the alerts and logs. – schroeder Oct 09 '17 at 09:19
  • It is all too common that a UBA system ends up creating more work for the SOC than it relives. But it all depends on what your goal is for the ML process. – schroeder Oct 09 '17 at 09:20
  • If I were to put my 'futurist' hat on, I can see a future where everyone has UBA systems and the hackers figure out the algorithms and devise ways to evade detection, just like the situation we are in for anti-virus. This does not make it useless, AV is still useful, but it is not a magic silver bullet to solve all problems everywhere. It has a place and a role to play. – schroeder Oct 09 '17 at 09:22
  • 1
    Right, and you can't just ignore a fire alarm because that could be exploited. Nice example. Also what exactly does UBA stand for ? – Marco Neves Oct 09 '17 at 09:33
  • 1
    UBA = "User Behaviour Analytics" Now more accurately known as UEBA = "User and Entity Behaviour Analytics", but the older term is more widely known – schroeder Oct 09 '17 at 09:35