In Windows any change in the system status, whether a regular system activity or some system fault is termed as an Event. Whenever, an event takes place in a system, the system components create an Event Message describing about the event. The event messages thus generated are stored in a storage area in the form of log entries. This entire procedure is termed as Event Logging. The Event Log file is a regular file with .evt file format. It contains event message and all other information related to event, such as event type, event status, event severity, event ID and much more. These logs can be modified by attaching the event messages. The system component which is responsible for editing event messages for event logging is a Log Client.
Event Logs play a very crucial role in modern days IT systems, especially at enterprise level where large networks are managed. Since Event logs are collected in real time by the system components, they serve as an excellent source of information to monitor systems and overall networks. Apart from this, the information obtained from event logs are used in log analysis which is very helpful during audit procedure and during management of IT infrastructure.
Syslog, earlier implemented for BSD UNIX; is a flexible event logging protocol that works well with several operating systems and can be implemented on different types of network devices, such as switches, printers, routers, firewalls, etc. To use this protocol, the log client must create a Syslog event message to log an event which is then send to Syslog server.
The UDP communication protocol is followed between the client and the Syslog server. UDP is a fast and appropriate communication protocol for large IT systems with several network nodes. Therefore, Syslog is considered as the ideal protocol for building a centralized logging infrastructure. Log monitoring tools and event log analyzer that are developed on the concept of centralized logging system all use the Syslog event logging protocol.
Due to the growing importance of event logs, especially at enterprise levels, new and advanced event log analyzer are being built, which will be capable of effectively monitoring both system based and web server based events. In addition, the advanced event log analyzer even implements event correlation, filtering and consolidation properties.
MANAGEENGINE
ventLog Analyzer, a log management software for SIEM, offers in-depth analytical capability to enhance network security with its predefined reports and real-time alerts. It also collects, monitors, correlates, and archives Windows event logs, syslogs, network devices logs, application logs, and more.
- In-depth auditing capabilities
- Augmented threat intelligence
- Comprehensive log management
- High-speed log processing
- Built-in incident management
SOLARWINDS
Perform Windows Event Log monitoring, analysis, and alerting with SolarWinds Log Analyzer.
- Monitor critical Windows Event Log data in real time to aid in troubleshooting
- Tag and filter your log data to help find entries faster
- See if an event contributed to performance issues
EVENTLOGXP
Windows event log analysis, view and monitor security, system, and other logs on Windows servers and workstations.
- Access Windows event logs and event log files on local and remote servers and workstations
- Support of both classic Windows NT event log format (EVT files) and new (Crimson) event log format (EVTX files)
- High performance – all events are loaded either into memory or into an optimized internal local database
- Active monitoring and alerting – get informed about problems immediately
- Event log consolidation – you can consolidate different events in one place
- Tabbed-document and multiple-document user interface depending on user preferences
- And More.
NETWRIX
Netwrix Event Log Manager surpasses Event Viewer by collecting, consolidating and archiving Windows server event logs and alerting on critical actions.
- Have all event log data in a single view
- Be notified about critical events
- Keep event logs stored for further analysis
- Forget about manual log monitoring processes
- Ensure compliance
- Get it free of charge
SECURE-BYTES
Secure Auditor most comprehensive software solutions consists of computer security database security, network security software, Network security tools, regulatory compliance, database auditing.
- User friendly and easy to use
- Helps in log viewing & log analyzing
- Hence this event log viewer helps user in better event log management
- Helps in enforcement of Regulatory Compliance Standards
- Detect problems with an authorization or access control implementation
- Helps to identify a possible brute force attack
- Helps to identify a possible password guessing by users
- Helps to identify a possible misuse of privileges
- Helps in identifying individual accountability
- And More.
NETVIZURA
NetVizura brings network monitoring solutions for bandwidth monitoring, network traffic investigation, analysis and reporting.
- Centralized Log Management
- Quick and Easy Search
- Great Configuration Abilities
Log Analysis for network security and Auditing
Log analysis is mainly performed to rectify or predetermine events that are security threats to a network. Since a vast amount of data gets collected in the form of event logs, processing and extracting useful information from them is not an easy task for administrators and IT managers, especially logistically. This is the reason advanced log analyzers are deployed in large organizations so as to consolidate event from network wide applications and log them in a centralized location for further processing and analysis. Log analysis is thus crucial when administrators attempt to decipher network and server issues. For optimum analysis results, an ideal log analyzer must be integrated with the following basic features:
- Real time monitoring: It is imperative to ensure that the logs being collected are of real time events for quick notification and proficient network analysis
- Centralized log monitoring: Event logs generated from individual computers are isolated from other events making event viewing difficult for analysts and administrators. Therefore, instead of viewing multiple events logs and tracking the events from multiple sources, it is ideal to have a central log monitoring system where consolidated data can be stored
- Remote log analysis: It is imperative to monitor logs, especially security event logs from a remote station and not just base network security on local security log analysis
- Critical event log alerts: The log information provided by Windows event log viewer is not detailed enough and highly cryptic, which makes their deciphering difficult. Therefore, modern log monitoring systems must come with applications that are capable of reading remote logs while notifying about critical events through alert generation
- Archiving event logs: During internal or external audits, event logs play a vital role as they form the main source of information about the entire network activity. Therefore, it becomes important to collect and retain the old log entries for longer duration so as to aid auditing. Automated log archiving systems help in this regard by eliminating events getting overwritten by more current logs
- Log file integrity: To ensure log security and preserve their integrity, it is best to store the event logs on a remote system where the chances of intrusion can be limited. Furthermore, this functionality is also important to curb network traffic generated due to logs and garbage data that comes from local machines
- Log filtering: an ideal log monitoring system must have the feature of log filtering so as to separate the critical events from unwanted traffic that emerges from the bulk of event logs generated in a network
- Monitoring web event logs: event logs generated from web server activities are highly crucial for network administration as they provide information about the local and remote web server
- Maintaining log database: Collecting thousands of event logs in searchable databases like SQL is advantageous in enterprise environment for facilitating centralized logging
- Event log reporting: Apart from event collection and log analysis, event log reporting is another key functionality that must be present in log analyzers. Automatic generation of reports containing data about the events, event types and cause of events help in analysis, administration, auditing and archiving
It can thus be concluded that accurate log analysis depends on various aspects and the limited features of Windows event log viewer makes administration of a large network complex. Log analysis and monitoring systems with advances features and functionalities prove to be the best solution for enterprise level administration.
Log File Analysis for optimum network security
IT businesses with dynamic structure require more continuity when it comes to analyzing security events and logs from an enterprise wide perspective. Processes such as documenting regulatory compliances, defining security policies and implementing network management technologies largely depend upon proper log file analysis. In order to fully realize a security event log management system, the entire logging system needs to be redefined. For effective log file analysis, event log management technologies have come a long way to incorporate better data analyzing capabilities.
Advancements in technologies have led to the development of log analysis systems which take into account multiple devices from multiple vendors across a network.
Security events deliver an encompassing overview of network security performance and therefore analyzing large amount of data collected over the network forms the basis of log management solutions. The major objectives of log file analysis can be summarized as follows:
- Through log analysis, critical events can be easily and automatically segregated from the normal data traffic
- Detailed and informative reports can be obtained and network security activities can be easily identified
- Log analysis proves beneficial for documenting the regulatory compliance and auditing practices within an organization
The need for log analysis
Security infrastructure within an enterprise forms a complex mesh with multiple software, services and applications configured to a network. As a result, the logs and the reporting structure become divergent with no set standards for data collection, storage and distribution. Devices such as firewalls, IDS and IPS generate massive amount so of data and streamlining the corresponding events is no simple task. Log analysis proves to be the optimum solution for securing a network by utilizing the information obtained from the event log files.
However, log file analysis of such heterogeneous data is only possible with generic type event messages instead of the usual cryptic event logs. Effective analysis further requires automatic aggregation of the event log data at a centralized location from where a consolidated view of the collected events can be obtained. Centralization of event logs in turn aids in better documenting and reporting practices, thereby resulting in faster log analysis and response time in case of security threats.
Centralized logging for event log consolidation and analysis
In a large network with several computers, thousands of peripherals, third party applications, scripts, certain Microsoft applications and custom applications, security is a big issue. In order to monitor and track down every such device or application status, what is required is powerful log analyzing systems. In this regard, Windows Event Log utility does not provide sufficient functionalities and many critical applications fail to write into the log records.
Nevertheless, the critical nature of such applications or services cannot be ruled out from the point of view of network security. Since, data collection from endpoint systems and devices with third party applications can impact an organization’s operations is a crucial way, systems which support event log consolidation are an absolute necessity.
Tools which function as event log viewer or analyzer must be capable of monitoring and consolidating any type of system data or application on a near real time basis. Event collection and correlation a large amount of data obtained from the network proves to be useful in accurate risk analysis.
What is event log consolidation?
Event log consolidation is the process of continuously monitoring data from the unprecedented amount of devices and applications that are configured to a network and collecting them in a centralized database for further analysis. The motive behind event log consolidation is to obtain useful and meaningful information from network firewalls, IPS, IDS, routers, etc instead of only cryptic data. Log records of security devices, network administration control devices like BDC, PDC and even Active Directory controllers provide important data to get a complete picture of the entire network.
Some of the critical information which can be obtained easily through event log consolidation is mentioned below:
- Tracking changes in the operating systems, files and Windows registry
- Monitoring USB device activity
- Tracking application specific log files
- Trending enterprise wide disk space usage
- Monitoring network connection and tracking critical services
- Tracking software installs and application usage
- Analyzing CPU and memory processes
Output generated from various devices of the network must be correlated with useless data traffic filtered out and only process the network critical data. Event log analysis tools which come with log consolidation and correlation features mostly do so through a GUI driven environment. Furthermore, such tools offer a centralized monitoring and control over all the peripheral outputs which are then consolidated and stored in a large database. This helps network administrators is parsing the collected log data for future analysis and auditing.
Windows event log types and their significance
Windows Event Log Viewer, the in-built event monitoring and collecting utility records all the application, system and security events and presents them in the form of log files. This default event log explorer application arms network administrators against the possible errors and problems in any part of the entire network topology. The event log store provides useful information for log analysis which is in turn used for auditing purpose. Event log explorer service such as Windows eventvwr.msc and LogWatNT.exe help in troubleshooting procedures by providing data collected from thousands of events that occur on a daily basis.
Similar event log monitoring and processing utilities used in Linux operating system like LogWatch helps in processing log files and include various other functionalities to generate detailed log reports, process particularly service specified event log files through filtering, search for specific log file in the entire log archive or log directory, save the output of the processed log and mail it to user specified addresses.
Since the network security has evolved with time, the need for better enterprise level event log monitoring systems have also increased. Event log explorer tools that are capable of providing event monitoring and administration solutions are fast replacing those only which basic functionalities of recording event logs.
Event log explorer tools are seen as a means of protecting the physical and logical assets in an organization from potential risks. Therefore, such tools must be equipped with features of not only collecting data from security events but also to collect generic data from other network endpoint devices such as routers, firewalls, intrusion detection systems and application servers.
Ideally, the new generation event log explorer applications are integrated with the capability of streamlining the events and collecting data from different sources to store in a large centralized database. Furthermore, these applications automatically correlate the collected event log data and provide system administrators with a consolidated view of the security policies and network performance. Event analysis and reporting part is also managed with event log explorer tools with focus on accurate distribution of the security information.
Therefore, it can be said that with thousands of devices configured to a network and millions of logs generated per day, event log explorers with excellent aggregation, correlation and consolidation properties are required. As networks evolve further integrating mergers and acquisitions, data selection, filtering, compression and reporting capabilities will take the center stage in log analysis.
Event Log Watch application to monitor system generated log files
Event Logwatch is a modular log analyzer that goes through all the system generated logs in a given period of time and creates detailed reports. This customizable, pluggable log monitoring system allows you to modify the system configuration files and also create new service filters for events. This analyzer can be run as a command-line tool to convert the raw log lines in structured format. The subscripts of this event log watch utility generate service specific output but ignore the time component. In other words, if you want to know the exact time of the recorded event, Logwatch will only report the event in a time range and for more details the raw log files will have to be searched for.
The main configuration file for Logwatch is located at /etc/logwatch/conf/logwatch.conf
Following is the syntax:
- logwatch [–detail level][–logfile log- file-group][–service service-name][–print][–mailto address][–archives][–range range][–debug level][–save file- name][–logdir directory][–hostname hostname][–help|–usage]
The different options available with the lowatch configuration files are as follows:
- Detail: Defines the detail level (high, med or low) of the generated reports
- Logfile: This option processes only the set of logfiles defined by log-file-group with all the contained services
- Service: this option processes only the services specified in service-name and therefore all the logfiles essential to process such services
- Print: This option prints the results on the screen
- Mailto: With this option all the generated reports can be mailed to specified recipients
- Archives: With this option logwatch conducts search in the log archives as well as the basic log files
- Range: This option allows users to specify the date range for log search and filtration
- Debug: This option is used by logwatch to debug the logfiles
- Save: This option helps in saving the logwatch output or the generated reports
- Logdir: With this option the logwatch looks in the log file directory instead of the default one
- Hostname: Uses hostname for reports in place of system’s hostname
- Usage/ Help: This option displays usage information
All the above mentioned options are available by default without any command line arguments upon running Logwatch utility.
Configuration structure of Logwtach
Logwatch consists of three configuration files, viz., /usr/share/logwatch/default.conf,
/usr/share/logwatch/dist.conf, and /etc/logwatch/conf. all these three configuration files have similar structures with the following subdirectories or components:
- Services: This subdirectory consists of all the configuration files that are specific to some service. By examining the contents of his directory, event log watch utility is able to determine the available service
- Logfiles: In this subdirectory, all the logfiles group subdirectory can be found which have information about multiple log files with similar format. It is possible to accesses a single logfile group configuration file by multiple services
- Logwatch.conf: All the defaults required to execute the event log watch utility is contained in this file. The parameters of this file can be overwritten or customized in command-line switches during invoking the logwatch executables
- Ignore.conf: This file specifies the regular expressions which suppress the matching line when matched with the event log watch output regardless of the service being executed
Customizing the configuration
It is possible to customize and modify the configuration file structure of logwatch so as to alter its default options, create new service filters and generate user specific reports. In order to customize the output of this event log watch utility, the variables in the directory file /etc/logwatch/conf directory have to be modified. Furthermore, modification can also be carried out by overwriting the default executable scripts in the directory file /etc/logwatch/scripts.
Event tracking through Event Log Analyzer Software
Event logs provide detailed insights into the Windows architecture and aids in effective monitoring and reporting. Windows event logs containing security and application data help administrators to quickly identify configuration changes and avert security or operation risks. However, for correct assessment of the logs, it is important for timely and competent event tracking.
Tracking down thousands of logs of a Windows system, correlating the data and then processing the information collected for further comparison is a lengthy procedure. Another important concern for effective event tracking is filtering the network traffic and processing information which helps in detecting changes in the network.
Event tracking tools combine all these aspects and offer functionalities such as event correlation, data processing and warehousing as well as an agile reporting capability. The ideal event tracking system must possess the following functionalities
- Data collection to support network auditing so as to adhere by compliance and security models
- Fast identification of changes that represent security risks to infrastructure and filtering out irrelevant data traffic
- Improved problem determination capability by integrating data changes to help in log analysis
- Minimizing system vulnerability to improve overall network security
- Alert creation capability on detection of system change or policy drift
- Ability to perform comparison processing both online and offline
- File system integrity checking through monitoring file attributes and contents
- Centralized console architecture and agent configuration for a single view of the entire network infrastructure along with log data summary
- Correlation between the actual and desired system state for exact log analysis
Proper event tracking system ensures better network and server security by recording logs of unnoticed or unauthorized changes. With timely monitoring and alert generation, it is possible for administrators to know about the system vulnerabilities and the precise times of change in network status. Such systems not only decrease the overall cost of IT infrastructure but also helps in policing the configuration management process.
Windows event log types and their significance
In Windows based systems, any event that occurs gets recorded in a log entry with the help of the Event Log Service. An event could be termination of an application, detection of a new hardware, starting a service, a new web page accessed, an error that occurred during an ongoing process and so on. The Event Log Service records all these types of events in the form of log entries on the basis of three major categories, namely, Application event logs, System event logs and Security event logs.
Windows event log types
Windows log files can be categorized on the basis of the event types recorded. Following are the details:
- Application Event Logs: These log files contain information about the program that run in a system, for example, a database program that encountered an error.
- Security Event Logs: These log files contain data related to resource use like creating or deleting files or CPU job scheduling. These logs also store data about security events such as valid or invalid logon attempts.
- System Event Logs: These log files contain information about the Windows operating system components, such as driver startup failure. System events are predetermined by Windows to be logged as system components.
In addition to these three basic event log types, there are other log files recoded by Windows Event Log Service that is mentioned below:
- Directory Service Logs: These log files contain information about the events or related activities that take place in Windows Active Directory or any such LDAP complaint services.
- File Replication Service (FRS) Logs: All the system based replication activities are recorded in the FRS log files.
- DNS Server: All types of DNS activities including DNS queries and DNS responses can be obtained from these log files.
Types of events
The events logged into the log files can be further classified into the following types on the basis of their severity:
- Information: An event that describes the successful completion of a task is categorized under this event type. Example of information could be successful loading of a network drive.
- Warning: An event that indicates a possible occurrence of problem, even though not significant comes under this event type category. Example of warning could be when the system disk space starts to run low.
- Error: An event that describes an error or significant problem in a program or application comes under the category of error type events. Example of error type event could be a failure of a critical task like service failure on startup.
- Success Audit (Security logs): The events that describe successful completion of audited security events fall under this category. A valid user logon is an example of success audit event type.
- Failure Audit (Security logs): Events that describe failure of an audited security event is categorized as failure audit event type. Example of a failure audit could be when a user is unable to access a network drive.
Windows offer an in-built utility called the Event Viewer (eventvwr.msc) which lets users view the logged events generated by a system on a daily basis. Hence, event viewer displays windows log files which contain information about your system, hardware and software components. Using these Windows log files, it is possible to gain insight about the system and network performance, thereby enforcing better security. Event logs generated by a system are of great assistance to network administrators in monitoring security events of local or remote computers, identifying the source of system problems and predicting the potential risks.
Web Log Viewer For log Analysis of Web Server Events
Web logs are access log files for web servers which contain information about the web access requests along with data about the web IP address, HTTP version, referrer page, browser type, timestamps, etc. the format used in web access log files is also termed as the W3C web server format. The W3C event logs for a web server creates records of the different elements found in a web page like image files, wave file, text files, etc. Other than that, these logs also contain page access details, user access details, number of visits on a web page, etc.
W3C extended log file format
W3C extended log file format is the default log file format used by IIS (Internet Information Services). These log files contains data such as date and time on which web activity occurred, IP address of clients that made a web access request, authenticated user name who accessed the server, internet service name and instance number, server IP address, HTTP status code, protocol version, site last visited by the user and much more.
W3C centralized logging
W3C centralized logging is a global configuration for FTP, SMTP, NNTP web servers so that all the web sites write their data into a single log file. The format used to store this data is the W3C extended log file format. This type of logging improves server scalability where large number of sites is hosted. By enabling the W3C centralized logging feature, every site on a particular server writes its log data into a central log file.
Web log viewer
A generic web log viewer acts as a W3C log analysis tool to process log file data written in the W3C web server format. In addition to data processing, web log viewers also generate dynamic statistics of the events along with detailed event reports. W3C event log viewers have the capacity of parsing the data, aggregating them and generating filtered reports through an interactive graphical interface. Web log viewer tools generally support multiple platforms such as Windows, Linux, Mac OS, Solaris, UNIX and Open BSD among others. Such tools track all the log file fields and combine the data from different sites and perform dynamic filtering.
Web log viewing tools enable site administrators and server managers to analyze the logs of IIS web servers and save the generated reports in different file formats like csv, html, txt, etc. by using these log analyzer tools, reviewing web server log reports get simpler thereby aiding in better network and server management tasks.