Logscape Support

Members Login
Username 
 
Password 
    Remember Me  
Logscape Support -> logscape -> Best way to extract pipe delimitted data
Post Info TOPIC: Best way to extract pipe delimitted data
neilson9


Senior Member

Status: Offline
Posts: 100
Date:
Best way to extract pipe delimitted data
Permalink  
 

Hi Max,

Interestingly we see the '|' delimiter being used quite a lot, especially when people are formatting their own log based event data. I guess it looks a bit like a column ;)

As you rightly say you could use a synthetic groovy script like option1 below however its not going to be as fast as option 2:

Option 1: groovy-script: return { msg.split("\|")[1] }

Option 2: split,\|,1
   (set the source field to be message)

 

Why? Option 1 is going to execute a groovy-script (which can be pretty fast), then the groovy script will use a pattern match to do the split. Option 2 just scans the line and returns the correct item. In terms of performance its going to be about 10x faster.

 

I Hope that helps,

Cheers N.

 

 



-- Edited by neilson9 on Monday 15th of October 2012 02:00:20 PM

__________________
max


Member

Status: Offline
Posts: 7
Date:
Permalink  
 

Hi,

We use a fairly generic datatype as we tend to differently formatted data all ending up in the same log file.

So given that i have a datatype with three non-synth fields:

date, time, data 

what would be the best way to create synthetic fields for all of the data between the pipes in the below line?

I currently create a synth, name instrumentation:

instrumentation - (**)

I then do a string.split("\\")[n] to extract the field i'm after.

Is there a better way.

10/15/12 09:30:18.043 INFO: [ProcessWrapper] [STDOUT] 2012-10-15 09:30:18,042 [latest-cache-serviceWorker:0] INFO  instrumentation - |4f9a85b2-b015-469c-ac5f-1daeec87713f|fd552222-8ceb-4f8f-9638-aef9a9ab5171|1|METRIC|StorageNodeAggregationTime|0



__________________
max


Member

Status: Offline
Posts: 7
Date:
Permalink  
 

Are you saying that substring,\|,1 will do the split for me and give me back the string at index 1 in an array?


__________________
neilson9


Senior Member

Status: Offline
Posts: 100
Date:
Permalink  
 

Hi Max,

Sorry about the typo - instead of using substring I should have entered 'split'.

i.e. Option 2: split,\|,1
   (set the source field to be message)

 

Cheers,

N

ps. Ive edited the previous posting.



__________________
Page 1 of 1  sorted by
 
Quick Reply

Please log in to post quick replies.
Logscape Support -> logscape -> Best way to extract pipe delimitted data


Create your own FREE Forum
Report Abuse 		Powered by ActiveBoard