Sorry I may be asking a silly question but I wrote a regex to capture fields in windows logs and I was hoping to use it to create fields in Logascape but for the life of me I can't where I need to put the regex to generate those fields
My regex is (?:\{"([0-9]+?), ([0-9]+?), "(.+?)", "(.+?)", "(?:.+?)", "(.+?)\n(?:[a-zA-Z0-9:,"\n \s\f\t\-\{\}\.\(\)]+?)New\sLogon:\s*(?:.+?)\n\s*Account\sName:\s*(.+?)\n(?:[a-zA-Z0-9:,"\n \s\f\t\-\{\}\.\(\)]+?)Source\sNetwork\sAddress:\s*([0-9\.]{7,15})\n(?:[a-zA-Z0-9:,"\n \s\f\t\-\{\}\.\(\)]+?)"\})
I tested in regex101.com and it works fine, my log records are like the below, I use {" as record delimiter and that work fine
This the ouput I would like, (my regex does correctly capture them)
time stamp
event id
status
tym
1435760640000
4624
Success
Security
An account was successfully logged on
INNOIWINTEL189$
135.75.26.116
I have highlighted in bold the fields that the regex captures, how do I make Logscape show those fields?
Below there is my windows log I have high lighted the values with the highlighted fields that are of interest for me and that are captured by the regex
{"1435760640000, 4624, "Success", "Security", "Microsoft-Windows-Security-Auditing", "An account was successfully logged on.
This event is generated when a logon session is created. It is generated on the computer that was accessed.
The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).
The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.
The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested." "}
Dump the data in and hit transform. All of the data should come in to the NewField column, every new line character will start a new line.
First of, GUID was automatically pulled out by Logscape as a key value pair - which is a good example of how Key Value pairs come in.
To get Login ID, add a new field (click the Plus icon on the left hand side) -
Change the name of NewField1 to ID
In Synth Source, type NewField
In Synth Expression, type Logon ID: (*)
Click OK
You should now have a white NewField, a blue ID field and a green GUID field
So rather than 1 mega regex, you create lots of little one's for each element.
I hope that helps. If you do decide to get the Windows App, most of the data should come in as KV pairs, but you may still need to use regex for some advanced stuff. Any questions, shout.
Your explanations and the videos in the documentation clarified heaps so really thanks for all the help, I have still an hurdle left,
I can get now the values on the first line of my records, but I cannot get the values of the other lines
I can capture the values in bold, cut I cannot capture the account name, that apperas several lines below, still they are all part of a single record/message.
{"1435760640000, 4624, "Success", "Security", "Microsoft-Windows-Security-Auditing", "An account was successfully logged on.
Interesting... Do you have this problem on the data types screen? Or does it work in the data types screen and then not actually in Logscape?
Remember, the Data Types screen doesn't do half of what Logscape actually does to the data, so normally data behaves better in the search.
When I put your data into the Data Types screen, I've put it in like so.
Notice the extra space after each line break, that keeps it on the same message line. So NewField has the entirety of the text.
Is that how it looks when Logscape ingests it?
Pattern: (**)
{"1435760640000, 4624, "Success", "Security", "Microsoft-Windows-Security-Auditing", "An account was successfully logged on.
Account Name: -
Account Domain: -
Logon ID: 0x0
New Logon:
Security ID: S-1-5-21-1039142878-192558930-806110124-105839
Account Name: INNOIWINTEL189$
When I did that I could get the data into synthetics ( account name needed Account Name: ([a-zA-Z0-9$]+) because it appears twice - so I would expect to see it work out in the wild.
If you get it to look like that in the Types screen but not out in the wild, it may be useful to upload a small sample, in case it's something irritating like BOM encoding.
Sorry for the late reply, I tried to attach a sample of the log but I couldn't anyway to me it looks the format on file is like the one I used on screen for testing so with all the line spaces etc.
Could you kindly tell me which segment of the regex you used in each synthetic field tested?
Because i need to confess I'm getting mighty confused with the part that there is different behavior between the data types screen and the search
Example I used for the time stamp (?:\{"([0-9]+?),
For even ID I can use , ([a-zA-Z0-9$]+)
But i just can't capture the Account name and the IP
Dump this XML into notepad, save as TEST.config and deploy it to your environment. It will give you a data type with the examples. Let me know if that helps
