Unable to send logs to GrayLog - Multiple Route

Post a different question
8
responses

Hi Folks,

I have following nxlog.conf which works fine.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\DNSIN.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output fileout>
    Module          om_file
    File            'C:\DNSLogs\DNSOUT.log'
    #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
</Output>

<Route parse_multiline>
    Path filein => process-buffer => fileout
</Route>

Now, I am trying to send the data to our log collector GrayLog & added output & route

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\MGTDCP03_DN.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output fileout>
    Module          om_file
    File            'C:\DNSLogs\MGTDCP03_DNSOUT-2.log'
    #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
</Output>

<Route parse_multiline>
    Path filein => process-buffer => fileout
</Route>

<Output gelf>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP
    <Exec>
      $gl2_source_collector = '${sidecar.nodeId}';
      $collector_node_id = '${sidecar.nodeName}';
    </Exec>
</Output>

<Route to_graylog>
  Path filein => process-buffer => gelf
</Route>

However, I start getting error msgs in nxlog.log. I have tried some troubleshooting steps but so far I am not able to get around the problem. Appreciate if you can help me resolve this issue.

2020-05-15 05:43:17 WARNING stopping nxlog service
2020-05-15 05:43:17 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 05:43:18 INFO nxlog-ce-2.10.2150 started
nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58
couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
invalid character: '
' (0xd)

2020-05-15 05:44:17 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58; couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 05:44:17 WARNING stopping nxlog service
2020-05-15 05:44:17 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 05:44:18 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58; couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 05:44:18 ERROR cannot add processor module 'process-buffer' to route 'to_graylog' because it is already added to route 'parse_multiline', you should define another instance at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 ERROR module 'gelf' has configuration errors, not adding to route 'to_graylog' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 ERROR route to_graylog is not functional without output modules, ignored at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 WARNING not starting unused module gelf
2020-05-15 05:44:18 INFO nxlog-ce-2.10.2150 started
AskedMay 15, 2020 - 7:52am

Answer (1)

Hello,

Two points to look at:
- you need to define one more pm_buffer module because first one already used by other route;
- something wrong with your Exec directive in gelf module, I'm not sure what you are trying to achieve using variables like this.

Regards, Arch

AnsweredMay 15, 2020 - 9:04am

Comments (7)

  • navdeepsingh83's picture

    Arch, i have sorted the issue with pm_buffer. Actually these configurations are pushed via graylog side car, and the variable populates hostname & collector node id automatically.

    Basically, I have 2 nxlog.conf file and they both work fine. One is for collection specific windows security event, and another one is for ingesting windows dns debug log with detailed option enabled.

    Now, when i try to combine both these files into 1 file, i receive several errors.

    Here is file 1 [working fine]

    define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Input eventlog>
        Module im_msvistalog
        PollInterval 1
        SavePos True
        ReadFromLast True

        #Channel System
    Query <QueryList>\
    <Query Id="0">\
    <Select Path="Security">*[System[(EventID=4625)]]</Select>\
    <Select Path="Security">*[System[(EventID=4740)]]</Select>\
    <Select Path="Security">*[System[(EventID=4771)]]</Select>\
    <Select Path="Security">*[System[(EventID=4720)]]</Select>\
    <Select Path="Security">*[System[(EventID=4722)]]</Select>\
    <Select Path="Security">*[System[(EventID=4723)]]</Select>\
    <Select Path="Security">*[System[(EventID=4724)]]</Select>\
    <Select Path="Security">*[System[(EventID=4725)]]</Select>\
    <Select Path="Security">*[System[(EventID=4726)]]</Select>\
    <Select Path="Security">*[System[(EventID=4738)]]</Select>\
    <Select Path="Security">*[System[(EventID=4767)]]</Select>\
    <Select Path="Security">*[System[(EventID=4781)]]</Select>\
    <Select Path="Security">*[System[(EventID=4772)]]</Select>\
    <Select Path="Security">*[System[(EventID=4624)]]</Select>\
    <Select Path="Security">*[System[(EventID=4647)]]</Select>\
    <Select Path="Security">*[System[(EventID=4625)]]</Select>\
    <Select Path="Security">*[System[(EventID=4778)]]</Select>\
    <Select Path="Security">*[System[(EventID=4779)]]</Select>\
    <Select Path="Security">*[System[(EventID=4800)]]</Select>\
    <Select Path="Security">*[System[(EventID=4801)]]</Select>\
    <Select Path="Security">*[System[(EventID=4802)]]</Select>\
    <Select Path="Security">*[System[(EventID=4803)]]</Select>\
    <Select Path="Security">*[System[(EventID=4731)]]</Select>\
    <Select Path="Security">*[System[(EventID=4727)]]</Select>\
    <Select Path="Security">*[System[(EventID=4754)]]</Select>\
    <Select Path="Security">*[System[(EventID=4735)]]</Select>\
    <Select Path="Security">*[System[(EventID=4737)]]</Select>\
    <Select Path="Security">*[System[(EventID=4755)]]</Select>\
    <Select Path="Security">*[System[(EventID=4734)]]</Select>\
    <Select Path="Security">*[System[(EventID=4730)]]</Select>\
    <Select Path="Security">*[System[(EventID=4758)]]</Select>\
    <Select Path="Security">*[System[(EventID=4732)]]</Select>\
    <Select Path="Security">*[System[(EventID=4728)]]</Select>\
    <Select Path="Security">*[System[(EventID=4756)]]</Select>\
    <Select Path="Security">*[System[(EventID=4733)]]</Select>\
    <Select Path="Security">*[System[(EventID=4729)]]</Select>\
    <Select Path="Security">*[System[(EventID=4769)]]</Select>\
    <Select Path="Security">*[System[(EventID=4768)]]</Select>\
    <Select Path="Security">*[System[(EventID=4757)]]</Select>\
    <Select Path="Security">*[System[(EventID=5141)]]</Select>\
    <Select Path="Security">*[System[(EventID=5137)]]</Select>\
    <Select Path="Security">*[System[(EventID=4763)]]</Select>\
    <Select Path="Security">*[System[(EventID=4749)]]</Select>\
    <Select Path="Security">*[System[(EventID=4753)]]</Select>\
    <Select Path="Security">*[System[(EventID=5139)]]</Select>\
    <Select Path="Security">*[System[(EventID=5137)]]</Select>\
    <Select Path="Security">*[System[(EventID=5136)]]</Select>\
    </Query>\
    </QueryList>
</Input>

<Output gelf>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP
    <Exec>
      # These fields are needed for Graylog
      $gl2_source_collector = '${sidecar.nodeId}';
      $collector_node_id = '${sidecar.nodeName}';
    </Exec>
</Output>


<Route route-1>
  Path eventlog => process-buffer => gelf
</Route>

    Here is file 2[working fine]

    define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_tcp
    Module          im_file
    File            'C:\DNSLogs\Server_DN.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output fileout>
    Module          om_file
    File            'C:\DNSLogs\Server_DNSOUT-2.log'
    #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
</Output>

<Output gelf>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP
    <Exec>
      $gl2_source_collector = '${sidecar.nodeId}';
      $collector_node_id = '${sidecar.nodeName}';
    </Exec>
</Output>

<Route to_graylog>
  Path filein => process-buffer => gelf
</Route>

<Route parse_multiline>
    Path filein => fileout
</Route>

    Here is merge of file 1 & 2 [not working]

    define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Input eventlog>
        Module im_msvistalog
        PollInterval 1
        SavePos True
        ReadFromLast True

        #Channel System
    Query <QueryList>
    <Query Id="0">
    <Select Path="Security">*[System[(EventID=4625)]]</Select>
    <Select Path="Security">*[System[(EventID=4740)]]</Select>
    <Select Path="Security">*[System[(EventID=4771)]]</Select>
    <Select Path="Security">*[System[(EventID=4720)]]</Select>
    <Select Path="Security">*[System[(EventID=4722)]]</Select>
    <Select Path="Security">*[System[(EventID=4723)]]</Select>
    <Select Path="Security">*[System[(EventID=4724)]]</Select>
    <Select Path="Security">*[System[(EventID=4725)]]</Select>
    <Select Path="Security">*[System[(EventID=4726)]]</Select>
    <Select Path="Security">*[System[(EventID=4738)]]</Select>
    <Select Path="Security">*[System[(EventID=4767)]]</Select>
    <Select Path="Security">*[System[(EventID=4781)]]</Select>
    <Select Path="Security">*[System[(EventID=4772)]]</Select>
    <Select Path="Security">*[System[(EventID=4624)]]</Select>
    <Select Path="Security">*[System[(EventID=4647)]]</Select>
    <Select Path="Security">*[System[(EventID=4625)]]</Select>
    <Select Path="Security">*[System[(EventID=4778)]]</Select>
    <Select Path="Security">*[System[(EventID=4779)]]</Select>
    <Select Path="Security">*[System[(EventID=4800)]]</Select>
    <Select Path="Security">*[System[(EventID=4801)]]</Select>
    <Select Path="Security">*[System[(EventID=4802)]]</Select>
    <Select Path="Security">*[System[(EventID=4803)]]</Select>
    <Select Path="Security">*[System[(EventID=4731)]]</Select>
    <Select Path="Security">*[System[(EventID=4727)]]</Select>
    <Select Path="Security">*[System[(EventID=4754)]]</Select>
    <Select Path="Security">*[System[(EventID=4735)]]</Select>
    <Select Path="Security">*[System[(EventID=4737)]]</Select>
    <Select Path="Security">*[System[(EventID=4755)]]</Select>
    <Select Path="Security">*[System[(EventID=4734)]]</Select>
    <Select Path="Security">*[System[(EventID=4730)]]</Select>
    <Select Path="Security">*[System[(EventID=4758)]]</Select>
    <Select Path="Security">*[System[(EventID=4732)]]</Select>
    <Select Path="Security">*[System[(EventID=4728)]]</Select>
    <Select Path="Security">*[System[(EventID=4756)]]</Select>
    <Select Path="Security">*[System[(EventID=4733)]]</Select>
    <Select Path="Security">*[System[(EventID=4729)]]</Select>
    <Select Path="Security">*[System[(EventID=4769)]]</Select>
    <Select Path="Security">*[System[(EventID=4768)]]</Select>
    <Select Path="Security">*[System[(EventID=4757)]]</Select>
    <Select Path="Security">*[System[(EventID=5141)]]</Select>
    <Select Path="Security">*[System[(EventID=5137)]]</Select>
    <Select Path="Security">*[System[(EventID=4763)]]</Select>
    <Select Path="Security">*[System[(EventID=4749)]]</Select>
    <Select Path="Security">*[System[(EventID=4753)]]</Select>
    <Select Path="Security">*[System[(EventID=5139)]]</Select>
    <Select Path="Security">*[System[(EventID=5137)]]</Select>
    <Select Path="Security">*[System[(EventID=5136)]]</Select>
    </Query>
    </QueryList>
</Input>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\server_DN.log'
    SavePos         TRUE
  # ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output gelf-1>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP

    # These fields are needed for Graylog
  #  Exec gl2_source_collector = ${sidecar.nodeId};
  #  Exec collector_node_id = ${sidecar.nodeName};

</Output>

<Output gelf-2>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP

    # These fields are needed for Graylog
   # Exec gl2_source_collector = ${sidecar.nodeId};
   # Exec collector_node_id = ${sidecar.nodeName};

</Output>

<Route route-1>
  Path filein, eventlog => process-buffer => gelf-1, gelf-2
</Route>

    Here is the error log from nxlog

    2020-05-15 12:25:50 INFO nxlog-ce-2.10.2150 started
2020-05-15 12:25:50 INFO connecting to 172.17.1.87:5044
nxlog failed to start: config file tag nesting is too large at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:47
2020-05-15 12:27:54 WARNING stopping nxlog service
2020-05-15 12:27:54 WARNING nxlog-ce received a termination request signal, exiting...
nxlog failed to start: config file tag nesting is too large at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:47

    May 15, 2020 - 2:30pm
  • navdeepsingh83's picture

    Here is the nxlog.conf generated via graylog sidecar

    define ROOT C:\Program Files (x86)\nxlog



Moduledir %ROOT%\modules

CacheDir %ROOT%\data

Pidfile %ROOT%\data\nxlog.pid

SpoolDir %ROOT%\data

LogFile %ROOT%\data\nxlog.log

LogLevel DEBUG



<Extension logrotate>

    Module  xm_fileop

    <Schedule>

        When    @daily

        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);

     </Schedule>

</Extension>



<Processor process-buffer>

  Module pm_buffer

  Type Mem

  MaxSize 16384

</Processor>



<Extension gelfExt>

  Module xm_gelf

  # Avoid truncation of the short_message field to 64 characters.

  ShortMessageLength 65536

</Extension>



<Input eventlog>

        Module im_msvistalog

        PollInterval 1

        SavePos True

        ReadFromLast True



        #Channel System

    Query <QueryList>

    <Query Id="0">

    <Select Path="Security">*[System[(EventID=4625)]]</Select>

    <Select Path="Security">*[System[(EventID=4740)]]</Select>

    <Select Path="Security">*[System[(EventID=4771)]]</Select>

    <Select Path="Security">*[System[(EventID=4720)]]</Select>

    <Select Path="Security">*[System[(EventID=4722)]]</Select>

    <Select Path="Security">*[System[(EventID=4723)]]</Select>

    <Select Path="Security">*[System[(EventID=4724)]]</Select>

    <Select Path="Security">*[System[(EventID=4725)]]</Select>

    <Select Path="Security">*[System[(EventID=4726)]]</Select>

    <Select Path="Security">*[System[(EventID=4738)]]</Select>

    <Select Path="Security">*[System[(EventID=4767)]]</Select>

    <Select Path="Security">*[System[(EventID=4781)]]</Select>

    <Select Path="Security">*[System[(EventID=4772)]]</Select>

    <Select Path="Security">*[System[(EventID=4624)]]</Select>

    <Select Path="Security">*[System[(EventID=4647)]]</Select>

    <Select Path="Security">*[System[(EventID=4625)]]</Select>

    <Select Path="Security">*[System[(EventID=4778)]]</Select>

    <Select Path="Security">*[System[(EventID=4779)]]</Select>

    <Select Path="Security">*[System[(EventID=4800)]]</Select>

    <Select Path="Security">*[System[(EventID=4801)]]</Select>

    <Select Path="Security">*[System[(EventID=4802)]]</Select>

    <Select Path="Security">*[System[(EventID=4803)]]</Select>

    <Select Path="Security">*[System[(EventID=4731)]]</Select>

    <Select Path="Security">*[System[(EventID=4727)]]</Select>

    <Select Path="Security">*[System[(EventID=4754)]]</Select>

    <Select Path="Security">*[System[(EventID=4735)]]</Select>

    <Select Path="Security">*[System[(EventID=4737)]]</Select>

    <Select Path="Security">*[System[(EventID=4755)]]</Select>

    <Select Path="Security">*[System[(EventID=4734)]]</Select>

    <Select Path="Security">*[System[(EventID=4730)]]</Select>

    <Select Path="Security">*[System[(EventID=4758)]]</Select>

    <Select Path="Security">*[System[(EventID=4732)]]</Select>

    <Select Path="Security">*[System[(EventID=4728)]]</Select>

    <Select Path="Security">*[System[(EventID=4756)]]</Select>

    <Select Path="Security">*[System[(EventID=4733)]]</Select>

    <Select Path="Security">*[System[(EventID=4729)]]</Select>

    <Select Path="Security">*[System[(EventID=4769)]]</Select>

    <Select Path="Security">*[System[(EventID=4768)]]</Select>

    <Select Path="Security">*[System[(EventID=4757)]]</Select>

    <Select Path="Security">*[System[(EventID=5141)]]</Select>

    <Select Path="Security">*[System[(EventID=5137)]]</Select>

    <Select Path="Security">*[System[(EventID=4763)]]</Select>

    <Select Path="Security">*[System[(EventID=4749)]]</Select>

    <Select Path="Security">*[System[(EventID=4753)]]</Select>

    <Select Path="Security">*[System[(EventID=5139)]]</Select>

    <Select Path="Security">*[System[(EventID=5137)]]</Select>

    <Select Path="Security">*[System[(EventID=5136)]]</Select>

    </Query>

    </QueryList>

</Input>



<Extension multiline>

    Module          xm_multiline

    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/

</Extension>



<Input filein>

    Module          im_file

    File            'C:\DNSLogs\MGTDCP03_DN.log'

    SavePos         TRUE

  # ReadFromLast    FALSE

    InputType       multiline

</Input>



<Output gelf-1>

    Module om_tcp

    Host 172.17.1.87

    Port 5044

    OutputType  GELF_TCP



    # These fields are needed for Graylog

  #  Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0;

  #  Exec collector_node_id = Server01;



</Output>



<Output gelf-2>

    Module om_tcp

    Host 172.17.1.87

    Port 5044

    OutputType  GELF_TCP



    # These fields are needed for Graylog

   # Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0;

   # Exec collector_node_id = Server01;



</Output>



<Route route-1>

  Path filein, eventlog => process-buffer => gelf-1, gelf-2

</Route>

    May 15, 2020 - 2:32pm
  • navdeepsingh83's picture

    Thanks manuel for you input, I used , and the problem was resolved.

    Now the only problem that is left that I am getting error when use these exec statements.

    Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0; Exec collector_node_id = Server01;

    nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112
couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
syntax error, unexpected =, expecting (
2020-05-15 23:21:20 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112; couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; syntax error, unexpected =, expecting (
2020-05-15 23:21:20 WARNING stopping nxlog service
2020-05-15 23:21:20 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 23:21:21 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112; couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; syntax error, unexpected =, expecting (
2020-05-15 23:21:21 ERROR module 'gelf-1' has configuration errors, not adding to route 'route-1' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:130
2020-05-15 23:21:21 WARNING not starting unused module gelf-1
2020-05-15 23:21:21 INFO connecting to 172.17.1.87:5044
2020-05-15 23:21:21 INFO nxlog-ce-2.10.2150 started

    If change Exec block to something like this. This is mentioned in graylog

        <Exec>
      # These fields are needed for Graylog
      $gl2_source_collector = '${sidecar.nodeId}';
      $collector_node_id = '${sidecar.nodeName}';
    </Exec>

    I get the following error

    nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111
couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
invalid character: '
' (0xd)
2020-05-15 23:31:01 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111; couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 23:31:01 WARNING stopping nxlog service
2020-05-15 23:31:01 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 23:31:02 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111; couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 23:31:02 ERROR module 'gelf-1' has configuration errors, not adding to route 'route-1' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:132
2020-05-15 23:31:02 WARNING not starting unused module gelf-1
2020-05-15 23:31:02 INFO connecting to 172.17.1.87:5044
2020-05-15 23:31:02 INFO nxlog-ce-2.10.2150 started

    Not sure what is going on. I have the same block in other Graylog NXlog templates and it works fine. however it's not working in the new merged template.

    May 16, 2020 - 1:59am
  • Arkadiy's picture

    Hello,

    Could you please drop your current config?
    I've tried to find those errors using previously posted but not sure is this is the right one.

    Regards, Arch

    May 26, 2020 - 6:44am
  • navdeepsingh83's picture

    I was able to sort it out, however 2 issues remain.

    1. when using graylog 3.4.4 with variables to populate nodeId and hostname, nxlog fails to start. $gl2_source_collector = '${sidecar.nodeId}'; $collector_node_id = '${sidecar.nodeName}';

    2. i have separated the output for dns & windows event, however duplicate logs are being received.

      define ROOT C:\Program Files (x86)\nxlog define NOT_STARTING_WITH_DATE_REGEX /^(?!\d+\/\d+\/\d+).+/ define EMPTY_EVENT_REGEX /(^$|^\s+$)/ define DROP_DNSQUERYEVENT_REGEX /((NXDOMAIN)|(YXDOMAIN))/ define PUBLICIPV4 /\b(?!(10)|192.168|172.(2[0-9]|1[6-9]|3[0-2]))[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/ define DIRECTION /(Snd)/

      Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Input eventlog>
        Module im_msvistalog
        PollInterval 1
        SavePos True
        ReadFromLast True

        #Channel System
    <QueryXML>
    <QueryList>
    <Query Id="0">
    <Select Path="Security">*[System[(EventID=4625)]]</Select>
    <Select Path="Security">*[System[(EventID=4740)]]</Select>
    <Select Path="Security">*[System[(EventID=4771)]]</Select>
    <Select Path="Security">*[System[(EventID=4720)]]</Select>
    <Select Path="Security">*[System[(EventID=4722)]]</Select>
    <Select Path="Security">*[System[(EventID=4723)]]</Select>
    <Select Path="Security">*[System[(EventID=4724)]]</Select>
    <Select Path="Security">*[System[(EventID=4725)]]</Select>
    <Select Path="Security">*[System[(EventID=4726)]]</Select>
    <Select Path="Security">*[System[(EventID=4738)]]</Select>
    <Select Path="Security">*[System[(EventID=4767)]]</Select>
    <Select Path="Security">*[System[(EventID=4781)]]</Select>
    <Select Path="Security">*[System[(EventID=4772)]]</Select>
    <Select Path="Security">*[System[(EventID=4624)]]</Select>
    <Select Path="Security">*[System[(EventID=4647)]]</Select>
    <Select Path="Security">*[System[(EventID=4625)]]</Select>
    <Select Path="Security">*[System[(EventID=4778)]]</Select>
    <Select Path="Security">*[System[(EventID=4779)]]</Select>
    <Select Path="Security">*[System[(EventID=4800)]]</Select>
    <Select Path="Security">*[System[(EventID=4801)]]</Select>
    <Select Path="Security">*[System[(EventID=4802)]]</Select>
    <Select Path="Security">*[System[(EventID=4803)]]</Select>
    <Select Path="Security">*[System[(EventID=4731)]]</Select>
    <Select Path="Security">*[System[(EventID=4727)]]</Select>
    <Select Path="Security">*[System[(EventID=4754)]]</Select>
    <Select Path="Security">*[System[(EventID=4735)]]</Select>
    <Select Path="Security">*[System[(EventID=4737)]]</Select>
    <Select Path="Security">*[System[(EventID=4755)]]</Select>
    <Select Path="Security">*[System[(EventID=4734)]]</Select>
    <Select Path="Security">*[System[(EventID=4730)]]</Select>
    <Select Path="Security">*[System[(EventID=4758)]]</Select>
    <Select Path="Security">*[System[(EventID=4732)]]</Select>
    <Select Path="Security">*[System[(EventID=4728)]]</Select>
    <Select Path="Security">*[System[(EventID=4756)]]</Select>
    <Select Path="Security">*[System[(EventID=4733)]]</Select>
    <Select Path="Security">*[System[(EventID=4729)]]</Select>
    <Select Path="Security">*[System[(EventID=4769)]]</Select>
    <Select Path="Security">*[System[(EventID=4768)]]</Select>
    <Select Path="Security">*[System[(EventID=4757)]]</Select>
    <Select Path="Security">*[System[(EventID=5141)]]</Select>
    <Select Path="Security">*[System[(EventID=5137)]]</Select>
    <Select Path="Security">*[System[(EventID=4763)]]</Select>
    <Select Path="Security">*[System[(EventID=4749)]]</Select>
    <Select Path="Security">*[System[(EventID=4753)]]</Select>
    <Select Path="Security">*[System[(EventID=5139)]]</Select>
    <Select Path="Security">*[System[(EventID=5137)]]</Select>
    <Select Path="Security">*[System[(EventID=5136)]]</Select>
    </Query>
    </QueryList>
    </QueryXML>
</Input>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\MGTDCP03_DNS.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
    <Exec>
     if $raw_event =~ %NOT_STARTING_WITH_DATE_REGEX% drop();
     if $raw_event =~ %EMPTY_EVENT_REGEX% drop();
     if $raw_event =~ %DROP_DNSQUERYEVENT_REGEX% drop();
     if $raw_event =~ %DIRECTION% drop();
     if not ($raw_event =~ %PUBLICIPV4%) drop();
    </Exec>
</Input>

<Output gelf-1>
    Module om_tcp
    Host 172.17.1.87
    Port 5046
    OutputType  GELF_TCP

    <Exec>
      # These fields are needed for Graylog
     # $gl2_source_collector = '${sidecar.nodeId}';
     # $collector_node_id = '${sidecar.nodeName}';
    </Exec>

</Output>

<Output gelf-2>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP

    <Exec>
      # These fields are needed for Graylog.
     # $gl2_source_collector = '${sidecar.nodeId}';
     # $collector_node_id = '${sidecar.nodeName}';
    </Exec>

</Output>

<Route route-1>
  Path filein, eventlog => process-buffer => gelf-1, gelf-2
</Route>

    May 27, 2020 - 4:20pm
  • manuel.munoz's picture

    I would say you need to separate the routes...

    <Route route-1>
  Path filein => process-buffer-1 => gelf-1
  Path eventlog => process-buffer-2 => gelf-2
</Route>

    Also you may need to remove the empty Exec blocks...

        <Exec>
      # These fields are needed for Graylog
     # $gl2_source_collector = '${sidecar.nodeId}';
     # $collector_node_id = '${sidecar.nodeName}';
    </Exec>

    May 28, 2020 - 12:18am