3
responses

Hi. I am relatively new to nxlog and I hope the community can help me with my question.
I have on a Windows Server an Windows Firewall Log File and following nxlog configuration:

<Input in>
Module im_file
File "C:\Windows\system32\LogFiles\Firewall\pfirewall.log"
SavePos TRUE
ReadFromLast TRUE
parse_syslog_ietf();
</Input>

<Output out>
Module om_tcp
Host 3.125.146.97:514
#Exec to_syslog_ietf();
</Output>

<Route 1>
Path in => out
</Route>

So far so good. The logs I receive are now as following:

<13>1 2022-07-08T13:44:08+00:00 ec2-3-66-101-32 1 - - - 2022-07-08T15:44:17.208477+02:00 WINFS02 - - - [NXLOG@14506 EventReceivedTime="2022-07-08 15:44:17" SourceModuleName="in" SourceModuleType="im_file"] 2022-07-08 15:44:18 ALLOW TCP X.X.X.X Y.Y.Y.Y 43031 Z 0 - 0 0 0 - - - RECEIVE

Can I shorten the message, that I receive just the end of the message?

2022-07-08 15:44:18 ALLOW TCP X.X.X.X Y.Y.Y.Y 43031 Z 0 - 0 0 0 - - - RECEIVE

AskedJuly 8, 2022 - 4:02pm

Answer (1)

First, it sounds like you just want the log back in RAW format. If so, then remove the statement ‘parse_syslog_ietf()’

Per NXLog’s documentation, the ietf adds the following items in parsing.
*PRI: Message Priority
*VERSION: Syslog format version
*TIMESTAMP: YYYY-MM-DDTHH:MM:SS.000000Z (or specified time zone)
*HOSTNAME
*APP-NAME: Device or Application that generated the message
*PROCID: Process ID
*MSGID: Message type

From the examples in the documentation and my own experience, the RAW log is then attached to the end of the parsed information. – So, by not running the parser, you should be back where you wanted to be.

I am referencing: https://docs.nxlog.co/userguide/integrate/syslog.html

I hope I didn't misunderstand your question.

Comments (2)

  • NX_RAF's picture

    Hi Scott

    Thank you for your reply. Also with the deleted parse_syslog_ietf() statement I receive the same message.
    I there a possibility to manipulate the entries form the raw log file bevor sending them to the syslog server?

    Regards

  • Scott Kisely's picture

    Wish I had remembered this first. Once I recalled it, then finding the documentation for it was easy. It converts data back to the $raw_event format. There are multiple methods to make these work, so I provided a few references to let you play out your method. I only created the easiest example below, not necessarily the best.

    First, I knew better - every im_ (input module) has parsers built in by default. Removing the other line in my first suggestion, simply removed the preferred method of parsing to the default method by the module.

    Second, I forgot that I used this function before and pretty sure I could have saved you some time if I could have remembered this first. Within the xm_syslog module are the procedures 'to_snare()' and 'to_syslog_snare()'
    With the im_file module, I believe 'to_snare()' is correct. I know 'to_syslog_snare()' is tailored to im_msvistalog first and is compatible with others as well. Don't be afraid to try the other snare procedure if needed. Can't promise that this will work for you, but it did work for me in a scenario somewhat similar.

    In my example below, I called up the xm_syslog extension module. Then in the output section, did my statement to Execute the snare option which converts the log back to $raw_event format. By doing this in the output section, this gives you back the im_ and pm_ modules for purposes of parsing, filtering, and whatnot - to $raw_event format to be sent out. (I did use the 'to_syslog_snare()' in my example because it does have a broader set of windows event fields already defined. And, I think this is the one that worked in my scenario.)

    -- And I haven't remembered how pick out data, so just going back to raw format seemed the smarter choice for the moment - pending a sudden insight that I just don't have right now. (or someone else telling us how to do it)

    References:
    For the xm_syslog module info - https://docs.nxlog.co/refman/v5.5/xm/syslog.html
    There is a very clean Example (#2) of forwarding logs here - https://docs.nxlog.co/userguide/configure/forwarding-logs.html
    (My example below is a mesh of yours and it)
    You may like this method better, depends on your preference. It gets rid of tab characters and newline sequences from the $Message field. See Example #15 from this - https://docs.nxlog.co/userguide/integrate/windows-eventlog.html

    Any feedback once you test would be great so I can continue to learn as well.
    =-=-=-=
    <Extension syslog>
    Module xm_syslog
    </Extension>

    <Input in>
    Module im_file
    File "C:\Windows\system32\LogFiles\Firewall\pfirewall.log"
    SavePos TRUE
    ReadFromLast TRUE
    </Input>

    <Output out>
    Module om_tcp
    Host 3.125.146.97:514
    Exec to_syslog_snare();
    </Output>

    <Route 1>
    Path in => out
    </Route>