Send CSV to syslog server

Tags:

#1 joniro

Hi. I have a script that fetches S3 logs from Amazon, unpacks them and puts them them to folder. I'm using this NXlog script below.
Problem is that NXlog doesn't start and I get following errors:

status nxlog.service
● nxlog.service - NXLog daemon
Loaded: loaded (/usr/lib/systemd/system/nxlog.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Sat 2018-11-24 02:44:31 EET; 2s ago
Process: 10620 ExecStartPre=/opt/nxlog/bin/nxlog -v (code=exited, status=1/FAILURE)

Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG stopping EXTENSION modules
Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG stopping module csv
Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: INPUT
Nov 24 02:44:31 -fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: PROCESSOR
Nov 24 02:44:31 -fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: OUTPUT
Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: EXTENSION
Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG no entries found, not writing configcache.dat
Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG nxlog_shutdown() leave
Nov 24 02:44:31 fetcher systemd[1]: Unit nxlog.service entered failed state.
Nov 24 02:44:31 fetcher systemd[1]: nxlog.service failed.

2018-11-24 02:48:15 DEBUG Error in Exec block: [to_syslog_bsd();]
2018-11-24 02:48:15 ERROR [modules.c:489/nx_ctx_config_modules()] -;[module.c:1567/nx_module_parse_exec_block()] Couldn't parse Exec block at /opt/nxlog/etc/nxlog.conf:37;[expr-grammar.y:381/parser_do()] couldn't parse statement at line 37, character 21 in /opt/nxlog/etc/nxlog.conf;[expr.c:3359/nx_expr_statement_new_procedure()] procedure 'to_syslog_bsd()' does not exist or takes different arguments
2018-11-24 02:48:15 DEBUG nxlog_shutdown() enter
2018-11-24 02:48:15 DEBUG stopping INPUT modules
2018-11-24 02:48:15 DEBUG stopping module in
2018-11-24 02:48:15 DEBUG stopping PROCESSOR modules
2018-11-24 02:48:15 DEBUG stopping OUTPUT modules
2018-11-24 02:48:15 DEBUG stopping module out
2018-11-24 02:48:15 DEBUG stopping EXTENSION modules
2018-11-24 02:48:15 DEBUG stopping module csv
2018-11-24 02:48:15 DEBUG shutdown_modules: INPUT
2018-11-24 02:48:15 DEBUG shutdown_modules: PROCESSOR
2018-11-24 02:48:15 DEBUG shutdown_modules: OUTPUT
2018-11-24 02:48:15 DEBUG shutdown_modules: EXTENSION
2018-11-24 02:48:15 DEBUG no entries found, not writing configcache.dat
2018-11-24 02:48:15 DEBUG nxlog_shutdown() leave


########################################
# Global directives #
########################################
User nxlog
Group nxlog

LogFile /opt/nxlog/var/log/nxlog/nxlog.log
LogLevel debug
Moduledir /opt/nxlog/libexec/nxlog/modules/
define ROOT /opt/nxlog
CacheDir /opt/nxlog/var/log/nxlog/
Pidfile /opt/nxlog/var/log/nxlog/nxlog.pid
SpoolDir /opt/nxlog/var/log/nxlog/


########################################
# Modules #
########################################
<Extension csv>
Module xm_csv
</Extension>

<Input in>
Module im_file
File "/home/cust/*.csv"
ReadFromLast FALSE
SavePos FALSE
</Input>

<Output out>
Module om_udp
Host 10.10.10.10
Port 514
Exec to_syslog_bsd();
#Exec to_syslog_ietf();
</Output>

########################################
# Routes #
########################################
<Route 1>
Path in => out
</Route>

#2 hcsDeactivated Nxlog ✓
#1 joniro
Hi. I have a script that fetches S3 logs from Amazon, unpacks them and puts them them to folder. I'm using this NXlog script below. Problem is that NXlog doesn't start and I get following errors: status nxlog.service ● nxlog.service - NXLog daemon Loaded: loaded (/usr/lib/systemd/system/nxlog.service; enabled; vendor preset: disabled) Active: activating (auto-restart) (Result: exit-code) since Sat 2018-11-24 02:44:31 EET; 2s ago Process: 10620 ExecStartPre=/opt/nxlog/bin/nxlog -v (code=exited, status=1/FAILURE) Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG stopping EXTENSION modules Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG stopping module csv Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: INPUT Nov 24 02:44:31 -fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: PROCESSOR Nov 24 02:44:31 -fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: OUTPUT Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG shutdown_modules: EXTENSION Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG no entries found, not writing configcache.dat Nov 24 02:44:31 fetcher nxlog[10620]: 2018-11-24 02:44:31 DEBUG nxlog_shutdown() leave Nov 24 02:44:31 fetcher systemd[1]: Unit nxlog.service entered failed state. Nov 24 02:44:31 fetcher systemd[1]: nxlog.service failed. 2018-11-24 02:48:15 DEBUG Error in Exec block: [to_syslog_bsd();] 2018-11-24 02:48:15 ERROR [modules.c:489/nx_ctx_config_modules()] -;[module.c:1567/nx_module_parse_exec_block()] Couldn't parse Exec block at /opt/nxlog/etc/nxlog.conf:37;[expr-grammar.y:381/parser_do()] couldn't parse statement at line 37, character 21 in /opt/nxlog/etc/nxlog.conf;[expr.c:3359/nx_expr_statement_new_procedure()] procedure 'to_syslog_bsd()' does not exist or takes different arguments 2018-11-24 02:48:15 DEBUG nxlog_shutdown() enter 2018-11-24 02:48:15 DEBUG stopping INPUT modules 2018-11-24 02:48:15 DEBUG stopping module in 2018-11-24 02:48:15 DEBUG stopping PROCESSOR modules 2018-11-24 02:48:15 DEBUG stopping OUTPUT modules 2018-11-24 02:48:15 DEBUG stopping module out 2018-11-24 02:48:15 DEBUG stopping EXTENSION modules 2018-11-24 02:48:15 DEBUG stopping module csv 2018-11-24 02:48:15 DEBUG shutdown_modules: INPUT 2018-11-24 02:48:15 DEBUG shutdown_modules: PROCESSOR 2018-11-24 02:48:15 DEBUG shutdown_modules: OUTPUT 2018-11-24 02:48:15 DEBUG shutdown_modules: EXTENSION 2018-11-24 02:48:15 DEBUG no entries found, not writing configcache.dat 2018-11-24 02:48:15 DEBUG nxlog_shutdown() leave ######################################## # Global directives # ######################################## User nxlog Group nxlog LogFile /opt/nxlog/var/log/nxlog/nxlog.log LogLevel debug Moduledir /opt/nxlog/libexec/nxlog/modules/ define ROOT /opt/nxlog CacheDir /opt/nxlog/var/log/nxlog/ Pidfile /opt/nxlog/var/log/nxlog/nxlog.pid SpoolDir /opt/nxlog/var/log/nxlog/ ######################################## # Modules # ######################################## <Extension csv> Module xm_csv </Extension> <Input in> Module im_file File "/home/cust/*.csv" ReadFromLast FALSE SavePos FALSE </Input> <Output out> Module om_udp Host 10.10.10.10 Port 514 Exec to_syslog_bsd(); #Exec to_syslog_ietf(); </Output> ######################################## # Routes # ######################################## <Route 1> Path in => out </Route>

Can you try adding in the extension module for Syslog in the config? It looks like only CSV extension module is added.

<Extension _syslog>
    Module  xm_syslog
</Extension>

Thanks