Custom Log Parser

Network administrators are always in need of more information and insights from their log data. There are times when an IT administrator would identify some log information which is useful and would like to have it indexed automatically as a new field. Having more fields being indexed makes your log data more useful while conducting log forensics analysis and creating network security reports.

EventLog Analyzer allows administrators to create custom (new) fields or extract fields from raw logs by using the interactive Field Extraction UI to create regular expression (RegEx) patterns to help EventLog Analyzer to identify, parse and index these custom fields from new logs it receives from network systems and applications.

How to extract additional fields using EventLog Analyzer?

• Navigate to the Search tab and search for the logs from which fields need to be extracted. Click Create Additional Fields to view and extract fields.
Note: Alternatively, you can also extract additional fields while importing the log file.

• You can view the extracted field details in the Event Information window. If the required value is not parsed, you can extract further fields by clicking the Extract Additional Fields.

Specifying custom field values

There are two methods by which custom fields can be specified viz.

• Regex method
• Delimiter method

Regex method

• Provide a rule name.
• Select and click the word(s) in the message, to be extracted as a field.
• You can use the Auto Identify option to identify the fields automatically.

• Provide a name for this field. Optionally, specify the prefix and suffix to the field value.
• Click on Create Pattern to generate a parser rule pattern.

Adding prefix and suffix

• You can also include the prefix and/or suffix of a field value to improve precision. To include a prefix and/or suffix, click on the icon in the right corner of the Fields table and select the required option. Click Apply.
• For instance, consider the message : Successful Network Logon: User Name: sylvian Domain: ADVENTNET Logon ID: (0x0,0x6D51131) Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: SYLVIAN Logon GUID: - Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 192.168.113.97 Source Port: 0 22873
• The prefix Logon Type can be a static value as most of the logs will have the exact word as Logon Type where as Source Network Address can be dynamic as the logs may have different word(s) like, Source IP Address, Source Address, but with the same pattern.
• If the prefix and suffix are defined with exact match, the field extraction will be precise.
Note: An open attribute will not have a prefix or suffix.

Validating the pattern

A parser rule pattern is created using the field definition. You can edit the generated pattern manually, if you are familiar with regular expressions.

Validate link is used to test the generated pattern against the previous search results. You can manually check the suitability of the pattern by analyzing the 'Matched Log Messages' and 'Unmatched Log Messages' displayed.

• Click on Choose another pattern to choose a pattern from the list of patterns generated by the application.
• You can define any existing field matching criteria to apply the pattern for this specific log type.
• Save the pattern to extract the field(s) from the upcoming logs.

Delimiter method

• Provide a rule name.
• You can use the Delimiter to extract fields using delimiters such as Space, Comma, Tab, or Pipe.

• To save the created rule, click Save rule.