26

Bit of newbie at the whole forensics stuff - but I'm trying to find out what I should have in place before an attack. While there is no end of material on the internet about forensics from seizure onwards, I'm trying to find out more about how I can make a secure record of events (specifically webserver logs) of adequate quality to be considered as evidence.

There are vague references to non-volatile media, hashing and signatures in the stuff I've read; these certainly provide a means for demonstrating a consistent snapshot - but do not intrinsically provide a mechanism for proving the data has not changed between initial capture and the snapshot, e.g. I could take today's log files and do a search/replace to overwrite the date with something else before committing the snapshot.

How does an electronic signature prove the data has not been tampered with between the initial capture and the signing - does it just support the signers assertion?

Must the integrity verification method be implemented in real-time? E.g. it's not very practical to write data directly to DVD, at best a track at a time is as near to real-time as you can get - but at a huge performance penalty.

Any pointers on content suitable for a non-lawyer? (pref with a EU/UK bias).

Hendrik Brummermann
  • 27,118
  • 6
  • 79
  • 121
symcbean
  • 18,278
  • 39
  • 73
  • 2
    It seems like you are conserned specifically about how to create and preserve evidence for legal matters instead of just internal evidence. Why is this? Have you been tasked with this responsibility? If your employer has a legal department or a consulting lawyer have you spoken with them? – this.josh Jun 05 '11 at 07:04

3 Answers3

31

This is an excellent and important question. There are several important techniques to know about:

  • Remote logging. Rather than store the log entries on the webserver, the webserver should be configured to send each log entry over the network to a log server. The log server should be a custom machine, configured for a single use (log recording only), and hardened. You should carefully minimize who has access to the log server, firewall it off from the outside world, and make sure it is running a minimum of services. This will help prevent tampering with logs after they are generated.

  • Hash chaining. Another important technique is to use cryptographic methods to protect the integrity of log records, once they are stored. A log entry should be a pair Xn = (Mn, Tn), where Mn is the nth log message received, and where Tn = Hash(Xn-1) is the cryptographic hash of the last log entry. Use SHA256 or some other collision-resistant cryptographic hash function for this purpose. What this does is ensure that an attacker can't tamper with entries in the middle of the log, without being detected. The attacker can replace all the entries in the "chain", or throw away a suffix and replace them with something new, but this constraints what the attacker can do. Importantly, the crypto ensures that if you write any log entry to write-once media, then the attacker can not change any earlier log entry without being detected. That's huge.

    For more, read about secure timestamping using hash chaining.

  • Write-once media. If you have write-once storage media, you can write the logs to the write-once media, and that will prevent tampering after they are written. Unfortunately, the choices here aren't great. As far as I know, they're basically:

    • CD-ROMs and DVD-ROMS are write-once, and allow appending if you use TAO mode, but also slightly clunky.

    • A line printer, printing onto paper, is a surprisingly effective write-once medium, as long as you keep the paper supply fed.

    • PROMs are write-once, if you use a Manchester encoding. Unfortunately, PROMs have mostly been replaced by EEPROMs these days, so true PROMs are hard to find.

    • Wikipedia also has a list of write-once media.

    • SanDisk has a SD card with 1GB of write-once storage, which is specifically designed for forensics and log storage purposes.

    Write-once media complement hash chaining extremely well. What you can do is store all log entries on some ordinary storage medium (e.g., a hard drive), and then once an hour store the latest log entry (or just its hash) to write-once storage. This ensures that an attacker who compromises or otherwise tampers with the log server can only modify records going back an hour, as well as (of course) all future records, but not records that were logged more than an hour before the compromise. That's huge.

  • Replication. You can store multiple copies of the log entries on multiple servers, to provide redundancy and protect against tampering. An attacker who breaks into one server and tampers with it won't be able to tamper with the copy on the other servers.

    For this to effective, you need the servers to be independent, so that it is unlikely that an attacker can compromise them all. For instance, you might physically locate the servers in different locations or different machine rooms, set up so that no one person has physical access to all locations. You might have separate individuals administering them, so that no one person has log-in access to all of the servers. You might have one log server running on your system, and a replica hosted remotely (e.g., running in the cloud).

    Replication complements hash chaining very nicely. For instance, rather than replicating every log entry, you can replicate just 1 out of every m log entries. Also, if you don't want to expose private log data to some of the replicas (e.g., ones running in the cloud), you don't have to store the entire log entry: you can store just the hash Tn. This hash value reveals nothing about the log entries themselves, since cryptographic hash functions are one-way.

I'm sure there is lots more that could be said about this, but I hope this helps introduce you to several technological methods that are available to protect logs.

Ron
  • 3
  • 2
D.W.
  • 98,420
  • 30
  • 267
  • 572
  • 1
    Thanks to all who replied - as I expected nobody seems to have a solution to prevent the compromise of logs between the event and the capture of the log information - but there are means for establishing the integrity of files from a given point onwards - I expect that the problem comes down to establishing a point where the logs are enrypted / timestamped and demostrating the integrity of this process - I suppose one way to address this would be by a trusted third party holding private keys / hash snapshots. – symcbean Jun 06 '11 at 09:34
4

Usual disclaimer: I'm not a lawyer. Please seek legal advice for admissibility in a specific court case.

What I have been told by lawyers though is that logs are admissible in courts if they have been captured as part of the ordinary course of business. So you do not have to do anything specific for this.

You do have to satisfy the evidentiary requirements though that the other side may throw up. So having a chain of custody and evidence that the logs have not been tampered with maybe useful and increases the weight of the logs in arguing your point e.g. proving a specific user or machine was used in an incident. Chain of custody in layman's terms means being able to demonstrate the "seizure, custody, control, transfer, analysis, and disposition of evidence, physical or electronic". For forensics and especially in criminal investigations the authorities need to be able to show chain of custody for the evidence to be admissible in a court

For this exporting the logs from the source system to a central Security Event and Information Management (SIEM) system that the administrators do not have access to is a good measure. Archiving this to write once read many (WORM) media is also good. Some SIEM's will allow you to generate a cryptographic hash of the log with the timestamp and store this on a separate media or system which will allow you to prove by re-calculating the hash that the log has not been tampered with.

Obviously if you can hash and archive in real time or at least daily that will provide you the best argument in a court. But having it even if done monthly would provide you a better argument than not I would have thought.

Rakkhi
  • 5,783
  • 1
  • 23
  • 47
  • Please explain what a SEIM is. Please explain chain of custody and how it relates to legal evidence. – this.josh Jun 05 '11 at 07:00
  • @this.josh, OK, to read more about what a SIEM is, [click me](http://lmgtfy.com/?q=SIEM&l=1). – D.W. Jun 06 '11 at 17:39
  • @D.W. I was trying to encourage you to add the definition to your answer. At least expand the acronym so that readers will have a chance to further research the topic. Links can be valuable, but they can also decay over time. There is no guarantee that the link you post today will be available next year. If the question and answer stay relevant after the link is gone, part of your answer will be missing. – this.josh Jun 06 '11 at 18:59
  • @this.josh, did you click the link? Somehow I don't think that one is going away. The top Google result for "SIEM" leads to a Wikipedia page, which I don't expect to disappear. And I think this demonstrates that readers can further research the topic quite easily enough with just the acronym. – D.W. Jun 07 '11 at 07:33
  • 1
    @D.W. That link is blocked by my corporate web content filter. "Your request was categorized as: 'Humor/Jokes'" – this.josh Jun 07 '11 at 15:38
  • @this.josh, OK, try it from home, and I think you'll see what I'm referring to. – D.W. Jun 07 '11 at 19:04
  • @D.W. Your link appears to have no value. It is equivelent to saying "google SIEM". If you want IT Security progress out of Beta, try to make your comments more constructive. [StackOverflow has actually banned the use of lfgtfy](http://meta.stackexchange.com/questions/15650/ban-lmgtfy-let-me-google-that-for-you-links). I'm not suggesting that IT Security will or should ban lmgtfy, but I think it demonstrates that lmgtfy lacks value. – this.josh Jun 08 '11 at 06:04
  • @this.josh, sorry if it was inappropriate. I'm making a serious point, which is that @Rakkhi's answer seems fine. Surely it isn't the purpose of the IT Security site to spoonfeed answers that already exist one Wikipedia or Google search away. I encourage all our readers to Google terms they don't know. If they can't be bothered to Google it first, I think it is disrespectful to expect us to explain it to them or to criticize @Rakkhi for not explaining it. Rather than criticizing @Rakkhi, I encourage you to do the research and improve @Rakkhi's answer yourself. – D.W. Jun 09 '11 at 07:15
  • 1
    @D-W @this-josh I have expanded SIEM and chain of custody and added links. Hopefully that helps. Sorry for the delay in responding, been overseas with work and there seemed is an excellent accepted answer above :) – Rakkhi Jun 09 '11 at 16:37
  • @D-W I thought that @Rakkhi 's answer was good, but I thought it could be better. I assume that people can search for the term SIEM, but that Rakkhi could produce a more relevant and valuable explanation for SIEM within the context of his answer. I didn't edit the post because I thought it was polite to give Rakkhi the opportunity to edit it himself. I have nothing at stake here. It's not my question and I haven’t provided a competing answer. My questions were meant to be editing suggestions. – this.josh Jun 09 '11 at 19:32
1

Snare allows you to transmit an MD5 hash appended to every message via standard Syslog. You can receive this using pretty much any SIEM. For the most part SIEM's in the top tier all support hashing of retained logs. This is in main part for searching and indexing, but are possible to use as a component of the chain-of-custody.

In combination with a strong separation of duties and a WORM archival of the retained and checksummed logs you've got a good chunk of it. Without WORM archival (not every runs Solaris after all) you still have your backup tapes that can be used for the same purpose.

The tapes themselves, if encrypted, and managed by another individual can assist in building that strong chain-of-custody as well. At that point, archival documentation is essential, most companies have requirements for removal and passing their backup tapes with a strong chain of custody for legal reasons.

With those pieces in place you can have a strong set of evidence. Once you get the lawyers involved, well, that's a different story.

Ori
  • 2,757
  • 1
  • 15
  • 29