How to Capture Specific Data
Hello All. I am at my wits end here. I am trying to capture specific data in a line of text so I can display it in a Dashboard.
I am trying to capture the data from a .txt file. In the file the line looks like this: Aug 09 09:00:08.076 CCSPAVCS01 VCS: [0x0000068c] ThrowingIncrementPolicy::increment1. newActiveCalls = 33 maxCalls = 220
The data I want is "newActiveCalls = 33". Basically, I am interested in all occurrences of that.
The section in NxLog for the log (called "messages.txt") basically looks like this: <Input ccsp_logs> Module im_file File "C:\temp\messages.txt" Exec parse_syslog(); Exec to_json(); </Input>
The output looks like this: {"EventReceivedTime":"2018-08-13 11:25:52","SourceModuleName":"ccsp_logs","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","Hostname":"CCSPAVCS01","EventTime":"2018-08-09 09:00:08","SourceName":"VCS","Message":"[0x0000068c] ThrowingIncrementPolicy::increment1. newActiveCalls = 33 maxCalls = 220 "}
As you can see, the data I want (newActiveCalls = 33) is encapulated in the "Message" field created by NxLog. I have tried numerous things (to many to list here). Does anyone know a way to capture that specific data so I may extract the numeral (in this case, 33; which is the important part) to create a quantified graphic for my servers? (In this case, I am using Kibana. The end result being a graph showing: Server-1 with 33 active calls, Server-2 with 20 active calls, etc.)
Thank you all for your time and help.
If you are still wanting the output in JSON where the entire $Message field is just the numerical value, you will need to use REGEX and capture groups.
Below is an example assuming that the input file is just the one type of event entry. If you have other types you will need to add additional if statements to drop or manage the events as needed.
The log_info() line is just so I can see the $Message field while nxlog is running in the foreground (-f) to make sure the regex works and is useful while troubleshooting things like this.
<Input ccsp_logs>
Module im_file
File 'capturegroup.txt'
ReadFromLast False
SavePos False
Exec parse_syslog();
<Exec>
if $raw_event =~ /^.+newActiveCalls\s=\s(\d+)\s.+/ $Message = $1;
</Exec>
Exec log_info("IN MSG : " + $Message);
Exec to_json();
</Input>
