3
responses

Hello - I am testing nxlog to send windows based syslogs to papertrail. The following code seems to produce the logs, however, the log header entries are logging with (date, external ip address, "logger:")  on each line. To be consistent with the other logs being collected, would like these headers to be (date, HOSTNAME of originator, Process originator) how can I change the header that nxlog is pushing out to papertrail? The information that I would like to populate the header is within the message body but am not clear how to populate the fields to the headers output?

Here is an example of the current output line:

Jan 11 10:25:30 159.88.217.201 logger:  good dog <14>1 2016-01-11T10:25:28.998751-05:00 Win8MacVM1 Microsoft-Windows-Kernel-General 3116 - [NXLOG@14506 Keywords="-9223372036854775792" EventType="INFO" EventID="1" ProviderGuid="{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}" Version="1" Task="0" OpcodeValue="0" RecordNumber="33715" ThreadID="3480" Channel="System" Domain="NT AUTHORITY" AccountName="SYSTEM" UserID="SYSTEM" AccountType="User" Opcode="Info" NewTime="2016-01-11T15:25:28.998234400Z" OldTime="2016-01-11T15:25:28.998234400Z" Reason="3" EventReceivedTime="2016-01-11 10:25:30" SourceModuleName="in" SourceModuleType="im_msvistalog"] The system time has changed to ‎2016‎-‎01‎-‎11T15:25:28.998234400Z from ‎2016‎-‎01‎-‎11T15:25:28.998234400Z.    Change Reason: System time adjusted to the new time zone.

Here is how I would like it to be:

Jan 11 10:25:30 Win8MacVM1 Microsoft-Windows-Kernel-General   good dog <14>1 2016-01-11T10:25:28.998751-05:00 Win8MacVM1 Microsoft-Windows-Kernel-General 3116 - [NXLOG@14506 Keywords="-9223372036854775792" EventType="INFO" EventID="1" ProviderGuid="{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}" Version="1" Task="0" OpcodeValue="0" RecordNumber="33715" ThreadID="3480" Channel="System" Domain="NT AUTHORITY" AccountName="SYSTEM" UserID="SYSTEM" AccountType="User" Opcode="Info" NewTime="2016-01-11T15:25:28.998234400Z" OldTime="2016-01-11T15:25:28.998234400Z" Reason="3" EventReceivedTime="2016-01-11 10:25:30" SourceModuleName="in" SourceModuleType="im_msvistalog"] The system time has changed to ‎2016‎-‎01‎-‎11T15:25:28.998234400Z from ‎2016‎-‎01‎-‎11T15:25:28.998234400Z.    Change Reason: System time adjusted to the new time zone.

Any clues from the community on how to accomplish this?

*** Code snippet to follow ***

## UDP Papertrail Test

define ROOT C:\Program Files (x86)\nxlog
define ROOT_STRING C:\Program Files (x86)\\nxlog
  
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

# Include fileop while debugging, also enable in the output module below
<Extension fileop>
    Module      xm_fileop
</Extension>
 
<Extension syslog>
    Module      xm_syslog
</Extension>

 
# Enable json extension
<Extension json>
    Module      xm_json
</Extension>
 
<Input in>
    Module      im_msvistalog
    Exec    to_json();

    
    ReadFromLast TRUE
    SavePos     TRUE
    Query       <QueryList>\
                    <Query Id="0">\
                        <Select Path="Application">*</Select>\
                        <Select Path="System">*</Select>\
                        <Select Path="Security">*</Select>\
                    </Query>\
                </QueryList>    

</Input>

 

<Output out>
   Module      om_udp
   Host xxxx.papertrailapp.com
    Port NNNNN
#Test to write unique entry in front of each message but did not change the actual header
   Exec to_syslog_ietf();\
$raw_event = " good dog "+$raw_event;
#Use the following line for debugging (uncomment the fileop extension above as well)
   Exec file_write("C:\\Program Files (x86)\\nxlog\data\\nxlog_output.log",  $raw_event);
</Output>

<Route 1>
    Path        in => out
</Route>

*** Code snippet end ***

AskedJanuary 11, 2016 - 4:43pm

Answer (1)

Hello,

the part "Jan 11 10:25:30 159.88.217.201 logger:  " is probably added by the Papertrail. You are sending them messages which aren't in a standard syslog format: they start with "good dog". I have never been using the Papertrail, but they will either expect the message to be in BSD syslog format (more probable) or IETF format. When this format is not detected, they prepend the "data logger:" part...

For BSD:

to_syslog_bsd(); # this takes current internal status of the message and formats it into the $raw_event field, which is then sent via UDP as is,

IETF:

to_syslog_ietf();

This command must be the last one changing $raw_event before sending the message out. If you want to add some text to the message, do so before calling to_syslog_*():

$Message = "nice dog " + $Message;
to_syslog_bsd();

M

 

Comments (2)

  • Zendog's picture

    Using The "to_syslog_bsd();" module seems to address the format processing issue with PaperTrail as you noted. It appears however that any changes to the message using IETF causes PaperTrail to freak out on the format again (regardless of where in the processing that it is edited) so I am still playing with this bit. However, using "to_syslog_bsd();" module with no edits is working as expected.

    Many thanks for the assist.