nxlog CE (windows) to syslog-ng (linux), invalid frame header, tcp connection dropped

Tags:

#1 toma27

I'm having some issues sending event logs from a windows 2012 R2 client using nxlog ce agent to a linux syslog-ng server.  The tcp connection appears to be getting dropped by syslog-ng due to an invalid frame header.

syslog-ng says:

2016-11-22T09:10:35+00:00 server syslog-ng[8317]: Invalid frame header; header=''
2016-11-22T09:10:35+00:00 server syslog-ng[8317]: Syslog connection closed; fd='8', client='AF_INET(ip.of.nxlog.client:55473)', local='AF_INET(0.0.0.0:6514)'
2016-11-22T09:10:36+00:00 server syslog-ng[8317]: Syslog connection accepted; fd='8', client='AF_INET(ip.of.nxlog.client:55474)', local='AF_INET(0.0.0.0:6514)'

nxlog says:

2016-11-22 15:24:27 ERROR om_tcp send failed; An established connection was aborted by the software in your host machine.  
2016-11-22 15:24:28 INFO connecting to ip.of.syslog-ng.server:6514
2016-11-22 15:24:28 INFO reconnecting in 1 seconds

I've tried several variations in the nxlog.conf file to work around this issue. 

Here is my nxlog.conf:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Input in>
    Module      im_msvistalog

# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
    ReadFromLast TRUE
    <QueryXML>
       <QueryList>
         <Query Id='1'>
            <Select Path='Application'>*</Select>
            <Select Path='Security'>*</Select>
            <Select Path='System'>*</Select>
         </Query>
       </QueryList>
   </QueryXML>
</Input>

<Processor eventlog_transformer>
  Module pm_transformer
  Exec $Hostname = hostname();
 OutputFormat syslog_rfc5424 
</Processor>

<Output out>
   Module om_tcp
   Host        ip.of.syslog-ng.server
   Port 6514
  
    Exec $raw_event = replace($raw_event, "\r\n", " ");
    Exec $raw_event = replace($raw_event, "\t", " ");
    #Exec to_syslog_ietf();
    #Exec        to_syslog_bsd();
</Output>

<Route 1>
    Path        in => eventlog_transformer => out
</Route>

Here is the syslog-ng configuration:

@version:3.5
@include "scl.conf"

# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
#       located in /etc/syslog-ng/conf.d/

#Options
#
## Warnings
#
#
options {
    create_dirs(yes);
    dir_perm(0755);
    perm(0644);

    log_msg_size(65536);
    log_fifo_size(10000);


    threaded(yes);
    ts_format(iso);

    keep_hostname(no);
    use_dns(no);
    dns_cache(no);
    use_fqdn(no);

    flush_lines(100);
    stats_freq(60);
    mark_freq(36400);

};

#Sources
source s_sys {
    system();
    internal();
};

source s_network {
    syslog( port(6514) flags(syslog-protocol)  transport("tcp") keep-alive(yes) so-keepalive(yes) so_rcvbuf(1073741824) log-fetch-limit(100) log-iw-size(100) max-connections(5000));
    syslog( port(514) transport("udp") so_rcvbuf(1073741824));

};

#Destinations
#destination d_all       { file("/var/log/logs/all_logs/$HOST/$HOST-$YEAR-$MONTH-$DAY-all_logs.log" create_dirs(yes)); };
destination d_all       { file("/var/log/logs/all_logs/$HOST/$HOST.log" create_dirs(yes)); };

#Filters

#Logs
#   { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); destination(d_all); };
log { source(s_network); destination(d_all); flags(flow-control); };

# Source additional configuration files (.conf extension only)
@include "/etc/syslog-ng/conf.d/*.conf"

 

I can get things to work using udp, but can't get it to work over tcp.  Has anyone else had any success getting nxlog to send events from windows to a syslog-ng server over tcp?

#2 b0ti Nxlog ✓ (Last updated )
#1 toma27
I'm having some issues sending event logs from a windows 2012 R2 client using nxlog ce agent to a linux syslog-ng server.  The tcp connection appears to be getting dropped by syslog-ng due to an invalid frame header. syslog-ng says: 2016-11-22T09:10:35+00:00 server syslog-ng[8317]: Invalid frame header; header='' 2016-11-22T09:10:35+00:00 server syslog-ng[8317]: Syslog connection closed; fd='8', client='AF_INET(ip.of.nxlog.client:55473)', local='AF_INET(0.0.0.0:6514)' 2016-11-22T09:10:36+00:00 server syslog-ng[8317]: Syslog connection accepted; fd='8', client='AF_INET(ip.of.nxlog.client:55474)', local='AF_INET(0.0.0.0:6514)' nxlog says: 2016-11-22 15:24:27 ERROR om_tcp send failed; An established connection was aborted by the software in your host machine.   2016-11-22 15:24:28 INFO connecting to ip.of.syslog-ng.server:6514 2016-11-22 15:24:28 INFO reconnecting in 1 seconds I've tried several variations in the nxlog.conf file to work around this issue.  Here is my nxlog.conf: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>     Module      xm_syslog </Extension> <Input in>     Module      im_msvistalog # For windows 2003 and earlier use the following: #   Module      im_mseventlog     ReadFromLast TRUE     <QueryXML>        <QueryList>          <Query Id='1'>             <Select Path='Application'>*</Select>             <Select Path='Security'>*</Select>             <Select Path='System'>*</Select>          </Query>        </QueryList>    </QueryXML> </Input> <Processor eventlog_transformer>   Module pm_transformer   Exec $Hostname = hostname();  OutputFormat syslog_rfc5424  </Processor> <Output out>    Module om_tcp    Host        ip.of.syslog-ng.server    Port 6514        Exec $raw_event = replace($raw_event, "\r\n", " ");     Exec $raw_event = replace($raw_event, "\t", " ");     #Exec to_syslog_ietf();     #Exec        to_syslog_bsd(); </Output> <Route 1>     Path        in => eventlog_transformer => out </Route> Here is the syslog-ng configuration: @version:3.5 @include "scl.conf" # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) #       located in /etc/syslog-ng/conf.d/ #Options # ## Warnings # # options {     create_dirs(yes);     dir_perm(0755);     perm(0644);     log_msg_size(65536);     log_fifo_size(10000);     threaded(yes);     ts_format(iso);     keep_hostname(no);     use_dns(no);     dns_cache(no);     use_fqdn(no);     flush_lines(100);     stats_freq(60);     mark_freq(36400); }; #Sources source s_sys {     system();     internal(); }; source s_network {     syslog( port(6514) flags(syslog-protocol)  transport("tcp") keep-alive(yes) so-keepalive(yes) so_rcvbuf(1073741824) log-fetch-limit(100) log-iw-size(100) max-connections(5000));     syslog( port(514) transport("udp") so_rcvbuf(1073741824)); }; #Destinations #destination d_all       { file("/var/log/logs/all_logs/$HOST/$HOST-$YEAR-$MONTH-$DAY-all_logs.log" create_dirs(yes)); }; destination d_all       { file("/var/log/logs/all_logs/$HOST/$HOST.log" create_dirs(yes)); }; #Filters #Logs #   { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); destination(d_all); }; log { source(s_network); destination(d_all); flags(flow-control); }; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf"   I can get things to work using udp, but can't get it to work over tcp.  Has anyone else had any success getting nxlog to send events from windows to a syslog-ng server over tcp?

The syslog-ng error message "Invalid frame header; header=" probably refers to framed IETF syslog. This can be produced by NXLog using a configuration as shown in this example. Here is the relevant part:


<Output out>
    Module	om_tcp
    Host	ip.of.syslog-ng.server
    Port	6514
    Exec	to_syslog_ietf();
    OutputType	Syslog_TLS
</Output>

It is recommended to generate the required format in your output block. While it will still work, using a processor module is deprecated.

Since NXLog supports Linux as well you might also consider installing it on the server side.