Syslog missing date time

Post a different question
4
responses

Hi, I am trying out nxlog (versus running a linux VM for syslog collection) and I my configuration works.
However, the nxlog file does not include the date time for each log entry.
I have experimented with bsd versus item with no change in result.
I read the manual but I am not seeing why nxlog would strip the date time form the received log message.
I am trying to figure out how to retain the date time values for each log line.

My config;
Panic Soft
#NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data

<Extension _syslog>
Module xm_syslog
</Extension>

<Input udp>
Module im_udp
Host 0.0.0.0
Port 514
Exec parse_syslog();
</Input>

<Output file>
Module om_file
File 'E:\smb\kam\syslog.txt'
<Exec>
if file->file_size() > 15M
{
$newfile = 'E:\smb\kam\syslog.old' + "." +
strftime(now(), "%Y%m%d%H%M%S");
file->rotate_to($newfile);
}
</Exec>
</Output>

<Route syslog_to_file>
Path udp => file
</Route>

<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>

<Extension _exec>
Module xm_exec
</Extension>

<Extension _fileop>
Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>

My nxlog file output with redactions <snip>;
PS E:\smb\kam> gc -tail 5 .\syslog.txt
<173>ulogd[869]: Blocked IN=eth1 OUT= MAC=<snip> SRC=185.153.<snip> DST=98.114.<snip> LEN=40 TOS=08 PREC=0x20 TTL=237 ID=4147 PROTO=TCP SPT=44338 DPT=23 SEQ=1112406427 ACK=0 WINDOW=1024 SYN URGP=0 MARK=0

Same syslog output from linux VM with redactions <snip>;
[root@penguin kam]# tail messages.txt
Feb 13 23:59:19 -05:00 ulogd[869]: Blocked IN=eth1 OUT= MAC=<snip> 184.105.<snip> 98.114.<snip> 40 00 0x00 247 54321 TCP 35457 30005 2026363779 0 65535 SYN

AskedFebruary 24, 2019 - 3:22pm

Answers (2)

It is possible that the time stamps may be arriving in a format that isn't expected and you may need to parse it manually. See the links below.

I would get the source event and compare it to the parsedate() function.
You can add one of the following to the Input block to grab the source.
file_write("C:\temp.log", $raw_event + "\n");
log_info($raw_event);

https://nxlog.co/documentation/nxlog-user-guide/timestamps.html#timestamps_parsing
https://nxlog.co/documentation/nxlog-user-guide/ref-lang.html#core_func_parsedate

AnsweredFebruary 25, 2019 - 4:06pm

Comments (1)

What you see in the output file is exactly what is being sent by the syslog device. parse_syslog() does not modify $raw_event so your om_file writes what im_udp receives.
If you want to fix the format you should add Exec to_syslog_bsd(); to your output module instance.

AnsweredFebruary 25, 2019 - 4:09pm

Comments (1)

  • kamsalisbury's picture

    Thank you b0ti, the added 'Exec to_syslog_bsd();' worked. Here is the file output now. Just what I need.

    PS E:\smb\kam> gc -tail 2 .\syslog.txt
    <173>ulogd[869]: Blocked IN=eth1 OUT= MAC=<snip> SRC=46.161.<snip> DST=98.114.<snip> LEN=40 TOS=00 PREC=0x00 TTL=247 ID=44504 PROTO=TCP SPT=45932 DPT=50404 SEQ=3580395651 ACK=0 WINDOW=1024 SYN URGP=0 MARK=0
    <173>Feb 27 21:21:00 192.168.<snip> ulogd[869]: Blocked IN=eth1 OUT= MAC=<snip> SRC=78.128.<snip> DST=98.114.<snip> LEN=40 TOS=00 PREC=0x00 TTL=243 ID=17887 PROTO=TCP SPT=47092 DPT=3200 SEQ=791584992 ACK=0 WINDOW=1024 SYN URGP=0 MARK=0

    I am new to using nxlog and now realize how powerful a utility it can be. I will be setting aside time to understand it more fully, now that I know it can totally replace a linux VM as a syslog collector for my needs.

    February 28, 2019 - 3:30am