Log management

Log management (LM) comprises an approach to dealing with large volumes of computer-generated log messages (also known as audit records, audit trails, event-logs, etc.).

Log management generally covers:[1]

  • Log collection
  • Centralized log aggregation
  • Long-term log storage and retention
  • Log rotation
  • Log analysis (in real-time and in bulk after storage)
  • Log search and reporting.

Overview

The primary drivers for log management implementations are concerns about security,[2] system and network operations (such as system or network administration) and regulatory compliance. Logs are generated by nearly every computing device, and can often be directed to different locations both on a local file system or remote system.

Effectively analyzing large volumes of diverse logs can pose many challenges, such as:

  • Volume: log data can reach hundreds of gigabytes of data per day for a large organization. Simply collecting, centralizing and storing data at this volume can be challenging.
  • Normalization: logs are produced in multiple formats. The process of normalization is designed to provide a common output for analysis from diverse sources.
  • Velocity: The speed at which logs are produced from devices can make collection and aggregation difficult
  • Veracity: Log events may not be accurate. This is especially problematic from systems that perform detection, such as intrusion detection systems.

Users and potential users of log management may purchase complete commercial tools or build their own log-management and intelligence tools, assembling the functionality from various open-source components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.[3]

Logging can produce technical information usable for the maintenance of applications or websites. It can serve:

  • to define whether a reported bug is actually a bug
  • to help analyze, reproduce and solve bugs
  • to help test new features in a development stage

Terminology

Suggestions were made to change the definition of logging. This change would keep matters both more pure and more easily maintainable:

  • Logging would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
  • Auditing, then, would involve data that is not immediately discardable. In other words: data that is assembled in the auditing process, is stored persistently, is protected by authorization schemes and is, always, connected to some end-user functional requirement.

Deployment life-cycle

One view of assessing the maturity of an organization in terms of the deployment of log-management tools might use successive levels such as:

  1. in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security-perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
  2. with increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security-perimeter.
  3. at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the enterprise — especially of those information-assets whose availability organizations regard as vital.
  4. organizations integrate the logs of various business-applications into an enterprise log manager for better value proposition.
  5. organizations merge the physical-access monitoring and the logical-access monitoring into a single view.
gollark: Interesting question.
gollark: x.com is hosted using Apache?!
gollark: Oops, wrong account.
gollark: In what way?
gollark: And yet.

See also

References
  1. (NIST), Author: Karen Kent; (NIST), Author: Murugiah Souppaya. "SP 800-92, Guide to Computer Security Log Management" (PDF). csrc.nist.gov.
  2. "Leveraging Log Data for Better Security". EventTracker SIEM, IT Security, Compliance, Log Management. Archived from the original on 28 December 2014. Retrieved 12 August 2015.
  3. "Top 5 Log Mistakes - Second Edition". Docstoc.com. Retrieved 12 August 2015.


This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.