Parsing XML style log

Post a different question
7
responses

Hi

I read the help but still don't understand how to send this log that would like to call impaxauditlog from a windowsbox text file (that is overwritten) to my syslog server. Below are some examples from the log and my current nxlog config setting.

Basically the entries take two lines one starts with !SYS and the other with <IHEYr4>. So based on the example the "!SYS 2015-02-23 11:51:49,478"  is the time the event was logged and "<TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp>" is the real event datetime that I would like use as syslog message datetime (you can ask AGFA for why the difference:) if this is not possible then of course I can settle with the 2015-02-23 11:51:49,478.

In essence the message itself should be everything that is between <IHEYr4></IHEYr4> tags BUT there is one big but - I need to parse the message a little and cut out everything that is between fist <CUID> and last </CUID> as there can be thousands of these and this is just wasting my diskspace. So the syslog message should look somethng like this:

2015-02-23T10:08:59+02:00 windowsbox impaxauditlog <IHEYr4><DicomQuery><CUID></CUID><SyntaxUID>LittleIndianImplicit</SyntaxUID></DicomQuery><Host>xx.xx.xx.xx</Host><TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp></IHEYr4>

Example:

!SYS 2015-02-23 11:51:49,478 - veebr 23 10:08:59 172.16.10.33 <?xml version="1.0" encoding="UTF-8" ?>
<IHEYr4><DicomQuery><CUID>1.3.12.2.1107.5.1.4.65588.30000015021704415668700013901</CUID><CUID>1.3.12.2.1107.5.1.4.65588.30000015021706453421800094763</CUID><SyntaxUID>LittleIndianImplicit</SyntaxUID></DicomQuery><Host>xx.xx.xx.xx</Host><TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp></IHEYr4>

Main part of my current config:

<Input in>

    Module      im_msvistalog
</Input>

<Extension syslog>
Module xm_syslog
</Extension>

<Output sslout>
Module om_ssl
Host xx.xx.xx.xx
Port 10514
Exec to_syslog_ietf();
CAFile %CERTDIR%/ca.cert.pem
CertFile %CERTDIR%/cert.pem
CertKeyFile %CERTDIR%/key.pem
AllowUntrusted TRUE
OutputType Syslog_TLS
</Output>

<Route 1>
    Path        in => sslout
</Route>

 

AskedFebruary 23, 2015 - 3:51pm

Answer (1)

You will probably want to use im_file with xm_multiline and then extract the fields such as:

Exec if $raw_event =~ /\<TimeStamp\>(.+)\<\/TimeStamp\>/ $EventTime = parsedate($1);

 

AnsweredFebruary 23, 2015 - 10:21pm

Comments (6)

  • bigfoot's picture

    So should this work or am I missing something?

    <Input in>
        Module      im_msvistalog
    </Input>

    <Extension syslog>

    Module xm_syslog
    </Extension>

    <Output sslout>
    Module om_ssl
    Host xx.xx.x.xx
    Port 10514
    Exec to_syslog_ietf();
    CAFile %CERTDIR%/ca.cert.pem
    CertFile %CERTDIR%/cert.pem
    CertKeyFile %CERTDIR%/key.pem
    AllowUntrusted TRUE
    OutputType Syslog_TLS
    </Output>

    <Extension multiline>
    Module xm_multiline
    HeaderLine /^!SYS/
    EndLine /^</IHEYr4>/
    </Extension>

    <Input filein>
    Module im_file
    File "C:\Program Files (x86)\Agfa\Sec\Audit\log\\*.log"
    SavePos FALSE
    ReadFromLast FALSE
    InputType multiline
    Exec if $raw_event !~ /^!SYS/ drop();
    Exec if $raw_event =~ /\<TimeStamp\>(.+)\<\/TimeStamp\>/ $EventTime = parsedate($1);
    </Input>

    <Route 1>
        Path        in => sslout
    </Route>
    <Route 2>
        Path        filein => sslout
    </Route>

    February 25, 2015 - 10:32am
  • bigfoot's picture

    okei, I got something working here but I still need to cut out all that is between first <CUID> and last </CUID> as I'm constantly getting the

    ERROR Syslog_TLS output is over the limit of 65000, will be truncated

    Is this possible?

    February 27, 2015 - 10:19am
  • adm's picture
    (NXLog)

    You either have very large messages or the xm_multiline HeaderLine/EndLine does not match properly and multiple messages get concatenated and reaching the limit.

    March 2, 2015 - 6:22pm
  • Aparna MANE's picture

    Hi,

    I am trying to cut the first field from syslog_bsd parsed output, wondering how i can do that, please see below

    <77>Jul 29 01:01:01 rhel66 run-parts(/etc/cron.hourly)[13751]: starting mcelog.cron
    <77>Jul 29 01:01:01 rhel66 run-parts(/etc/cron.hourly)[13781]: finished mcelog.cron

    need to cut <77> from the log

     

    Thanks,

    Aparna

    July 29, 2015 - 10:12am
  • adm's picture
    (NXLog)

    Please don't hijack other people's questions, you should create a new question if it is unrelated to the topic.

    Exec if $raw_event =~ /^\<\d+\>(.+)/ $raw_event = $1;

    July 29, 2015 - 11:00am