Logscape Support

Hi Skazal. 

I can see you already have a good understanding of how the data types page works. My answer is slightly verbose for the sake of clarity so I may be repeating a few things you already know. 

When creating a datatype you have two types of fields, you have the regular fields and the synthetics. The regular fields are mapped sequentially from the groups in your regular expression. The regular fields represent the structure of every log line in your file. Taking your example, the structure of your log file is DATE,TIME,THREAD,LEVEL,exTIME and MSG.  A correctly formatted log entry will always have this structure.  The regex pattern is used to capture this structure of your logfile. 

The synthetic fields are there to extract data that may not occur on every line, the variable part of your logs.  The synthetic fields do not have to occur on every line where as the regular fields must match each line for it to be processed. For example, the MSG field will always contain text but the SQLRESULT field will only contain a value if it finds/matches the sql results in the MSG field. If you take a look at the image I have attached, I've highlighted the synthetic fields and the regular fields

I have used this pattern to extract your log data 

(*) (*)\s+\[(**)\] (INFO|WARN|LEVEL|ERROR) (\d+)ms(**)

How to use the Synthetic Fields

1.) Create the sqlResult synth field to extract  the text  1144276771 46341656 1960505241 1369989566 775921648 1394433352 SELECT DISCOVERYRESOURCE_0

from the MSG field 

Name: sqlResult

synth source:  MSG 

synth expression: (**)SELECT DISCOVERYRESOURCE

The sqlResult field will be populated with a value after running the synth expression on the value held in the field referenced in the synth source. The first group matched is always used as the value. 

2.) Create the synths to extract the values held in sqlResult

With your particular data it was necessary to create the sqlResult synth field before attempting to extract the numerical values.  

Name: val1

synth source: sqlResult 

synth expression: (\d+) 

Val1 will apply the pattern (\d+) on the text 1144276771 46341656 1960505241 1369989566 775921648 1394433352 SELECT DISCOVERYRESOURCE_0 

to extract 1144276771

Name: val2

synth source: sqlResult 

synth expression: \d+ (\d+)

Val2 will match the second digit from the sqlResult and so on with the other values. 

The text diagram crudely expresses how each val1,val2 ,val3 obtain their values from a raw log line. 

RAW LOG DATA -> MSG  -> sqlResult    | -> val1 

                                                            |->val2 

                                                            |->val3 

3.) Synth Expressions

Synth experssions do not always have to have a synth source. A synth expression can perform numerical calculations fields, it can be a regex pattern, or a groovy and any of the Logscape text functions like split, substring. Here's an example synth expression that does a number conversion. 

Name: val1K

synth expression: jep:  val1 / 1024 

4.) Uploading the example datatype. 

You can see how I have put all of this together in the data type skazal which I will email to you. Navigate to the Configure/Backup page and click OVERWRITE to upload the skazal.config file. Once you have done that navigate to the Configure/Datatypes page and click open to search for the skazal datatype.