8
responses

Hi Folks,

I have following nxlog.conf which works fine.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\DNSIN.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output fileout>
    Module          om_file
    File            'C:\DNSLogs\DNSOUT.log'
    #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
</Output>

<Route parse_multiline>
    Path filein => process-buffer => fileout
</Route>

Now, I am trying to send the data to our log collector GrayLog & added output & route

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>

<Processor process-buffer>
  Module pm_buffer
  Type Mem
  MaxSize 16384
</Processor>

<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
</Extension>

<Input filein>
    Module          im_file
    File            'C:\DNSLogs\MGTDCP03_DN.log'
#   SavePos         TRUE
    ReadFromLast    FALSE
    InputType       multiline
</Input>

<Output fileout>
    Module          om_file
    File            'C:\DNSLogs\MGTDCP03_DNSOUT-2.log'
    #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
</Output>

<Route parse_multiline>
    Path filein => process-buffer => fileout
</Route>

<Output gelf>
    Module om_tcp
    Host 172.17.1.87
    Port 5044
    OutputType  GELF_TCP
    <Exec>
      $gl2_source_collector = '${sidecar.nodeId}';
      $collector_node_id = '${sidecar.nodeName}';
    </Exec>
</Output>

<Route to_graylog>
  Path filein => process-buffer => gelf
</Route>

However, I start getting error msgs in nxlog.log. I have tried some troubleshooting steps but so far I am not able to get around the problem. Appreciate if you can help me resolve this issue.

2020-05-15 05:43:17 WARNING stopping nxlog service
2020-05-15 05:43:17 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 05:43:18 INFO nxlog-ce-2.10.2150 started
nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58
couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
invalid character: '
' (0xd)

2020-05-15 05:44:17 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58; couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 05:44:17 WARNING stopping nxlog service
2020-05-15 05:44:17 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-15 05:44:18 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:58; couldn't parse statement at line 59, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
2020-05-15 05:44:18 ERROR cannot add processor module 'process-buffer' to route 'to_graylog' because it is already added to route 'parse_multiline', you should define another instance at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 ERROR module 'gelf' has configuration errors, not adding to route 'to_graylog' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 ERROR route to_graylog is not functional without output modules, ignored at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:65
2020-05-15 05:44:18 WARNING not starting unused module gelf
2020-05-15 05:44:18 INFO nxlog-ce-2.10.2150 started
AskedMay 15, 2020 - 7:52am

Answer (1)

Hello,

Two points to look at:
- you need to define one more pm_buffer module because first one already used by other route;
- something wrong with your Exec directive in gelf module, I'm not sure what you are trying to achieve using variables like this.

Regards, Arch

Comments (7)

  • navdeepsingh83's picture

    Arch, i have sorted the issue with pm_buffer. Actually these configurations are pushed via graylog side car, and the variable populates hostname & collector node id automatically.

    Basically, I have 2 nxlog.conf file and they both work fine. One is for collection specific windows security event, and another one is for ingesting windows dns debug log with detailed option enabled.

    Now, when i try to combine both these files into 1 file, i receive several errors.

    Here is file 1 [working fine]

    define ROOT C:\Program Files (x86)\nxlog
    
    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data
    LogFile %ROOT%\data\nxlog.log
    LogLevel INFO
    
    <Extension logrotate>
        Module  xm_fileop
        <Schedule>
            When    @daily
            Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
         </Schedule>
    </Extension>
    
    <Processor process-buffer>
      Module pm_buffer
      Type Mem
      MaxSize 16384
    </Processor>
    
    <Extension gelfExt>
      Module xm_gelf
      # Avoid truncation of the short_message field to 64 characters.
      ShortMessageLength 65536
    </Extension>
    
    <Input eventlog>
            Module im_msvistalog
            PollInterval 1
            SavePos True
            ReadFromLast True
    
            #Channel System
        Query <QueryList>\
        <Query Id="0">\
        <Select Path="Security">*[System[(EventID=4625)]]</Select>\
        <Select Path="Security">*[System[(EventID=4740)]]</Select>\
        <Select Path="Security">*[System[(EventID=4771)]]</Select>\
        <Select Path="Security">*[System[(EventID=4720)]]</Select>\
        <Select Path="Security">*[System[(EventID=4722)]]</Select>\
        <Select Path="Security">*[System[(EventID=4723)]]</Select>\
        <Select Path="Security">*[System[(EventID=4724)]]</Select>\
        <Select Path="Security">*[System[(EventID=4725)]]</Select>\
        <Select Path="Security">*[System[(EventID=4726)]]</Select>\
        <Select Path="Security">*[System[(EventID=4738)]]</Select>\
        <Select Path="Security">*[System[(EventID=4767)]]</Select>\
        <Select Path="Security">*[System[(EventID=4781)]]</Select>\
        <Select Path="Security">*[System[(EventID=4772)]]</Select>\
        <Select Path="Security">*[System[(EventID=4624)]]</Select>\
        <Select Path="Security">*[System[(EventID=4647)]]</Select>\
        <Select Path="Security">*[System[(EventID=4625)]]</Select>\
        <Select Path="Security">*[System[(EventID=4778)]]</Select>\
        <Select Path="Security">*[System[(EventID=4779)]]</Select>\
        <Select Path="Security">*[System[(EventID=4800)]]</Select>\
        <Select Path="Security">*[System[(EventID=4801)]]</Select>\
        <Select Path="Security">*[System[(EventID=4802)]]</Select>\
        <Select Path="Security">*[System[(EventID=4803)]]</Select>\
        <Select Path="Security">*[System[(EventID=4731)]]</Select>\
        <Select Path="Security">*[System[(EventID=4727)]]</Select>\
        <Select Path="Security">*[System[(EventID=4754)]]</Select>\
        <Select Path="Security">*[System[(EventID=4735)]]</Select>\
        <Select Path="Security">*[System[(EventID=4737)]]</Select>\
        <Select Path="Security">*[System[(EventID=4755)]]</Select>\
        <Select Path="Security">*[System[(EventID=4734)]]</Select>\
        <Select Path="Security">*[System[(EventID=4730)]]</Select>\
        <Select Path="Security">*[System[(EventID=4758)]]</Select>\
        <Select Path="Security">*[System[(EventID=4732)]]</Select>\
        <Select Path="Security">*[System[(EventID=4728)]]</Select>\
        <Select Path="Security">*[System[(EventID=4756)]]</Select>\
        <Select Path="Security">*[System[(EventID=4733)]]</Select>\
        <Select Path="Security">*[System[(EventID=4729)]]</Select>\
        <Select Path="Security">*[System[(EventID=4769)]]</Select>\
        <Select Path="Security">*[System[(EventID=4768)]]</Select>\
        <Select Path="Security">*[System[(EventID=4757)]]</Select>\
        <Select Path="Security">*[System[(EventID=5141)]]</Select>\
        <Select Path="Security">*[System[(EventID=5137)]]</Select>\
        <Select Path="Security">*[System[(EventID=4763)]]</Select>\
        <Select Path="Security">*[System[(EventID=4749)]]</Select>\
        <Select Path="Security">*[System[(EventID=4753)]]</Select>\
        <Select Path="Security">*[System[(EventID=5139)]]</Select>\
        <Select Path="Security">*[System[(EventID=5137)]]</Select>\
        <Select Path="Security">*[System[(EventID=5136)]]</Select>\
        </Query>\
        </QueryList>
    </Input>
    
    <Output gelf>
        Module om_tcp
        Host 172.17.1.87
        Port 5044
        OutputType  GELF_TCP
        <Exec>
          # These fields are needed for Graylog
          $gl2_source_collector = '${sidecar.nodeId}';
          $collector_node_id = '${sidecar.nodeName}';
        </Exec>
    </Output>
    
    
    <Route route-1>
      Path eventlog => process-buffer => gelf
    </Route>
    

    Here is file 2[working fine]

    define ROOT C:\Program Files (x86)\nxlog
    
    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data
    LogFile %ROOT%\data\nxlog.log
    LogLevel INFO
    
    <Extension logrotate>
        Module  xm_fileop
        <Schedule>
            When    @daily
            Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
         </Schedule>
    </Extension>
    
    <Processor process-buffer>
      Module pm_buffer
      Type Mem
      MaxSize 16384
    </Processor>
    
    <Extension gelfExt>
      Module xm_gelf
      # Avoid truncation of the short_message field to 64 characters.
      ShortMessageLength 65536
    </Extension>
    
    <Extension multiline>
        Module          xm_multiline
        HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
    </Extension>
    
    <Input filein>
        Module          im_tcp
        Module          im_file
        File            'C:\DNSLogs\Server_DN.log'
    #   SavePos         TRUE
        ReadFromLast    FALSE
        InputType       multiline
    </Input>
    
    <Output fileout>
        Module          om_file
        File            'C:\DNSLogs\Server_DNSOUT-2.log'
        #Exec    $raw_event = "-------------------------------------\n" + $raw_event;
    </Output>
    
    <Output gelf>
        Module om_tcp
        Host 172.17.1.87
        Port 5044
        OutputType  GELF_TCP
        <Exec>
          $gl2_source_collector = '${sidecar.nodeId}';
          $collector_node_id = '${sidecar.nodeName}';
        </Exec>
    </Output>
    
    <Route to_graylog>
      Path filein => process-buffer => gelf
    </Route>
    
    <Route parse_multiline>
        Path filein => fileout
    </Route>
    

    Here is merge of file 1 & 2 [not working]

    define ROOT C:\Program Files (x86)\nxlog
    
    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data
    LogFile %ROOT%\data\nxlog.log
    LogLevel INFO
    
    <Extension logrotate>
        Module  xm_fileop
        <Schedule>
            When    @daily
            Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
         </Schedule>
    </Extension>
    
    <Processor process-buffer>
      Module pm_buffer
      Type Mem
      MaxSize 16384
    </Processor>
    
    <Extension gelfExt>
      Module xm_gelf
      # Avoid truncation of the short_message field to 64 characters.
      ShortMessageLength 65536
    </Extension>
    
    <Input eventlog>
            Module im_msvistalog
            PollInterval 1
            SavePos True
            ReadFromLast True
    
            #Channel System
        Query <QueryList>
        <Query Id="0">
        <Select Path="Security">*[System[(EventID=4625)]]</Select>
        <Select Path="Security">*[System[(EventID=4740)]]</Select>
        <Select Path="Security">*[System[(EventID=4771)]]</Select>
        <Select Path="Security">*[System[(EventID=4720)]]</Select>
        <Select Path="Security">*[System[(EventID=4722)]]</Select>
        <Select Path="Security">*[System[(EventID=4723)]]</Select>
        <Select Path="Security">*[System[(EventID=4724)]]</Select>
        <Select Path="Security">*[System[(EventID=4725)]]</Select>
        <Select Path="Security">*[System[(EventID=4726)]]</Select>
        <Select Path="Security">*[System[(EventID=4738)]]</Select>
        <Select Path="Security">*[System[(EventID=4767)]]</Select>
        <Select Path="Security">*[System[(EventID=4781)]]</Select>
        <Select Path="Security">*[System[(EventID=4772)]]</Select>
        <Select Path="Security">*[System[(EventID=4624)]]</Select>
        <Select Path="Security">*[System[(EventID=4647)]]</Select>
        <Select Path="Security">*[System[(EventID=4625)]]</Select>
        <Select Path="Security">*[System[(EventID=4778)]]</Select>
        <Select Path="Security">*[System[(EventID=4779)]]</Select>
        <Select Path="Security">*[System[(EventID=4800)]]</Select>
        <Select Path="Security">*[System[(EventID=4801)]]</Select>
        <Select Path="Security">*[System[(EventID=4802)]]</Select>
        <Select Path="Security">*[System[(EventID=4803)]]</Select>
        <Select Path="Security">*[System[(EventID=4731)]]</Select>
        <Select Path="Security">*[System[(EventID=4727)]]</Select>
        <Select Path="Security">*[System[(EventID=4754)]]</Select>
        <Select Path="Security">*[System[(EventID=4735)]]</Select>
        <Select Path="Security">*[System[(EventID=4737)]]</Select>
        <Select Path="Security">*[System[(EventID=4755)]]</Select>
        <Select Path="Security">*[System[(EventID=4734)]]</Select>
        <Select Path="Security">*[System[(EventID=4730)]]</Select>
        <Select Path="Security">*[System[(EventID=4758)]]</Select>
        <Select Path="Security">*[System[(EventID=4732)]]</Select>
        <Select Path="Security">*[System[(EventID=4728)]]</Select>
        <Select Path="Security">*[System[(EventID=4756)]]</Select>
        <Select Path="Security">*[System[(EventID=4733)]]</Select>
        <Select Path="Security">*[System[(EventID=4729)]]</Select>
        <Select Path="Security">*[System[(EventID=4769)]]</Select>
        <Select Path="Security">*[System[(EventID=4768)]]</Select>
        <Select Path="Security">*[System[(EventID=4757)]]</Select>
        <Select Path="Security">*[System[(EventID=5141)]]</Select>
        <Select Path="Security">*[System[(EventID=5137)]]</Select>
        <Select Path="Security">*[System[(EventID=4763)]]</Select>
        <Select Path="Security">*[System[(EventID=4749)]]</Select>
        <Select Path="Security">*[System[(EventID=4753)]]</Select>
        <Select Path="Security">*[System[(EventID=5139)]]</Select>
        <Select Path="Security">*[System[(EventID=5137)]]</Select>
        <Select Path="Security">*[System[(EventID=5136)]]</Select>
        </Query>
        </QueryList>
    </Input>
    
    <Extension multiline>
        Module          xm_multiline
        HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
    </Extension>
    
    <Input filein>
        Module          im_file
        File            'C:\DNSLogs\server_DN.log'
        SavePos         TRUE
      # ReadFromLast    FALSE
        InputType       multiline
    </Input>
    
    <Output gelf-1>
        Module om_tcp
        Host 172.17.1.87
        Port 5044
        OutputType  GELF_TCP
    
        # These fields are needed for Graylog
      #  Exec gl2_source_collector = ${sidecar.nodeId};
      #  Exec collector_node_id = ${sidecar.nodeName};
    
    </Output>
    
    <Output gelf-2>
        Module om_tcp
        Host 172.17.1.87
        Port 5044
        OutputType  GELF_TCP
    
        # These fields are needed for Graylog
       # Exec gl2_source_collector = ${sidecar.nodeId};
       # Exec collector_node_id = ${sidecar.nodeName};
    
    </Output>
    
    <Route route-1>
      Path filein, eventlog => process-buffer => gelf-1, gelf-2
    </Route>
    

    Here is the error log from nxlog

    2020-05-15 12:25:50 INFO nxlog-ce-2.10.2150 started
    2020-05-15 12:25:50 INFO connecting to 172.17.1.87:5044
    nxlog failed to start: config file tag nesting is too large at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:47
    2020-05-15 12:27:54 WARNING stopping nxlog service
    2020-05-15 12:27:54 WARNING nxlog-ce received a termination request signal, exiting...
    nxlog failed to start: config file tag nesting is too large at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:47
    

  • navdeepsingh83's picture

    Here is the nxlog.conf generated via graylog sidecar

    define ROOT C:\Program Files (x86)\nxlog
    
    
    
    Moduledir %ROOT%\modules
    
    CacheDir %ROOT%\data
    
    Pidfile %ROOT%\data\nxlog.pid
    
    SpoolDir %ROOT%\data
    
    LogFile %ROOT%\data\nxlog.log
    
    LogLevel DEBUG
    
    
    
    <Extension logrotate>
    
        Module  xm_fileop
    
        <Schedule>
    
            When    @daily
    
            Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
    
         </Schedule>
    
    </Extension>
    
    
    
    <Processor process-buffer>
    
      Module pm_buffer
    
      Type Mem
    
      MaxSize 16384
    
    </Processor>
    
    
    
    <Extension gelfExt>
    
      Module xm_gelf
    
      # Avoid truncation of the short_message field to 64 characters.
    
      ShortMessageLength 65536
    
    </Extension>
    
    
    
    <Input eventlog>
    
            Module im_msvistalog
    
            PollInterval 1
    
            SavePos True
    
            ReadFromLast True
    
    
    
            #Channel System
    
        Query <QueryList>
    
        <Query Id="0">
    
        <Select Path="Security">*[System[(EventID=4625)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4740)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4771)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4720)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4722)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4723)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4724)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4725)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4726)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4738)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4767)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4781)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4772)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4624)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4647)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4625)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4778)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4779)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4800)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4801)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4802)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4803)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4731)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4727)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4754)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4735)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4737)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4755)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4734)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4730)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4758)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4732)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4728)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4756)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4733)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4729)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4769)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4768)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4757)]]</Select>
    
        <Select Path="Security">*[System[(EventID=5141)]]</Select>
    
        <Select Path="Security">*[System[(EventID=5137)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4763)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4749)]]</Select>
    
        <Select Path="Security">*[System[(EventID=4753)]]</Select>
    
        <Select Path="Security">*[System[(EventID=5139)]]</Select>
    
        <Select Path="Security">*[System[(EventID=5137)]]</Select>
    
        <Select Path="Security">*[System[(EventID=5136)]]</Select>
    
        </Query>
    
        </QueryList>
    
    </Input>
    
    
    
    <Extension multiline>
    
        Module          xm_multiline
    
        HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
    
    </Extension>
    
    
    
    <Input filein>
    
        Module          im_file
    
        File            'C:\DNSLogs\MGTDCP03_DN.log'
    
        SavePos         TRUE
    
      # ReadFromLast    FALSE
    
        InputType       multiline
    
    </Input>
    
    
    
    <Output gelf-1>
    
        Module om_tcp
    
        Host 172.17.1.87
    
        Port 5044
    
        OutputType  GELF_TCP
    
    
    
        # These fields are needed for Graylog
    
      #  Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0;
    
      #  Exec collector_node_id = Server01;
    
    
    
    </Output>
    
    
    
    <Output gelf-2>
    
        Module om_tcp
    
        Host 172.17.1.87
    
        Port 5044
    
        OutputType  GELF_TCP
    
    
    
        # These fields are needed for Graylog
    
       # Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0;
    
       # Exec collector_node_id = Server01;
    
    
    
    </Output>
    
    
    
    <Route route-1>
    
      Path filein, eventlog => process-buffer => gelf-1, gelf-2
    
    </Route>
    

  • navdeepsingh83's picture

    Thanks manuel for you input, I used , and the problem was resolved.

    Now the only problem that is left that I am getting error when use these exec statements.

    Exec gl2_source_collector = 32fe8b2d-5c52-4ecd-9e6f-bf99d93cd0f0; Exec collector_node_id = Server01;

    nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112
    couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
    syntax error, unexpected =, expecting (
    2020-05-15 23:21:20 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112; couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; syntax error, unexpected =, expecting (
    2020-05-15 23:21:20 WARNING stopping nxlog service
    2020-05-15 23:21:20 WARNING nxlog-ce received a termination request signal, exiting...
    2020-05-15 23:21:21 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:112; couldn't parse statement at line 112, character 31 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; syntax error, unexpected =, expecting (
    2020-05-15 23:21:21 ERROR module 'gelf-1' has configuration errors, not adding to route 'route-1' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:130
    2020-05-15 23:21:21 WARNING not starting unused module gelf-1
    2020-05-15 23:21:21 INFO connecting to 172.17.1.87:5044
    2020-05-15 23:21:21 INFO nxlog-ce-2.10.2150 started
    

    If change Exec block to something like this. This is mentioned in graylog

        <Exec>
          # These fields are needed for Graylog
          $gl2_source_collector = '${sidecar.nodeId}';
          $collector_node_id = '${sidecar.nodeName}';
        </Exec>
    

    I get the following error

    nxlog failed to start: Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111
    couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf
    invalid character: '
    ' (0xd)
    2020-05-15 23:31:01 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111; couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
    2020-05-15 23:31:01 WARNING stopping nxlog service
    2020-05-15 23:31:01 WARNING nxlog-ce received a termination request signal, exiting...
    2020-05-15 23:31:02 ERROR Couldn't parse Exec block at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:111; couldn't parse statement at line 113, character 67 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ';' (0xd)
    2020-05-15 23:31:02 ERROR module 'gelf-1' has configuration errors, not adding to route 'route-1' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:132
    2020-05-15 23:31:02 WARNING not starting unused module gelf-1
    2020-05-15 23:31:02 INFO connecting to 172.17.1.87:5044
    2020-05-15 23:31:02 INFO nxlog-ce-2.10.2150 started
    

    Not sure what is going on. I have the same block in other Graylog NXlog templates and it works fine. however it's not working in the new merged template.

  • Arkadiy's picture
    (NXLog)

    Hello,

    Could you please drop your current config?
    I've tried to find those errors using previously posted but not sure is this is the right one.

    Regards, Arch

  • navdeepsingh83's picture

    I was able to sort it out, however 2 issues remain.

    1. when using graylog 3.4.4 with variables to populate nodeId and hostname, nxlog fails to start. $gl2_source_collector = '${sidecar.nodeId}'; $collector_node_id = '${sidecar.nodeName}';

    2. i have separated the output for dns & windows event, however duplicate logs are being received.

      define ROOT C:\Program Files (x86)\nxlog define NOT_STARTING_WITH_DATE_REGEX /^(?!\d+\/\d+\/\d+).+/ define EMPTY_EVENT_REGEX /(^$|^\s+$)/ define DROP_DNSQUERYEVENT_REGEX /((NXDOMAIN)|(YXDOMAIN))/ define PUBLICIPV4 /\b(?!(10)|192.168|172.(2[0-9]|1[6-9]|3[0-2]))[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/ define DIRECTION /(Snd)/

      Moduledir %ROOT%\modules
      CacheDir %ROOT%\data
      Pidfile %ROOT%\data\nxlog.pid
      SpoolDir %ROOT%\data
      LogFile %ROOT%\data\nxlog.log
      LogLevel INFO
      
      <Extension logrotate>
          Module  xm_fileop
          <Schedule>
              When    @daily
              Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
           </Schedule>
      </Extension>
      
      <Processor process-buffer>
        Module pm_buffer
        Type Mem
        MaxSize 16384
      </Processor>
      
      <Extension gelfExt>
        Module xm_gelf
        # Avoid truncation of the short_message field to 64 characters.
        ShortMessageLength 65536
      </Extension>
      
      <Input eventlog>
              Module im_msvistalog
              PollInterval 1
              SavePos True
              ReadFromLast True
      
              #Channel System
          <QueryXML>
          <QueryList>
          <Query Id="0">
          <Select Path="Security">*[System[(EventID=4625)]]</Select>
          <Select Path="Security">*[System[(EventID=4740)]]</Select>
          <Select Path="Security">*[System[(EventID=4771)]]</Select>
          <Select Path="Security">*[System[(EventID=4720)]]</Select>
          <Select Path="Security">*[System[(EventID=4722)]]</Select>
          <Select Path="Security">*[System[(EventID=4723)]]</Select>
          <Select Path="Security">*[System[(EventID=4724)]]</Select>
          <Select Path="Security">*[System[(EventID=4725)]]</Select>
          <Select Path="Security">*[System[(EventID=4726)]]</Select>
          <Select Path="Security">*[System[(EventID=4738)]]</Select>
          <Select Path="Security">*[System[(EventID=4767)]]</Select>
          <Select Path="Security">*[System[(EventID=4781)]]</Select>
          <Select Path="Security">*[System[(EventID=4772)]]</Select>
          <Select Path="Security">*[System[(EventID=4624)]]</Select>
          <Select Path="Security">*[System[(EventID=4647)]]</Select>
          <Select Path="Security">*[System[(EventID=4625)]]</Select>
          <Select Path="Security">*[System[(EventID=4778)]]</Select>
          <Select Path="Security">*[System[(EventID=4779)]]</Select>
          <Select Path="Security">*[System[(EventID=4800)]]</Select>
          <Select Path="Security">*[System[(EventID=4801)]]</Select>
          <Select Path="Security">*[System[(EventID=4802)]]</Select>
          <Select Path="Security">*[System[(EventID=4803)]]</Select>
          <Select Path="Security">*[System[(EventID=4731)]]</Select>
          <Select Path="Security">*[System[(EventID=4727)]]</Select>
          <Select Path="Security">*[System[(EventID=4754)]]</Select>
          <Select Path="Security">*[System[(EventID=4735)]]</Select>
          <Select Path="Security">*[System[(EventID=4737)]]</Select>
          <Select Path="Security">*[System[(EventID=4755)]]</Select>
          <Select Path="Security">*[System[(EventID=4734)]]</Select>
          <Select Path="Security">*[System[(EventID=4730)]]</Select>
          <Select Path="Security">*[System[(EventID=4758)]]</Select>
          <Select Path="Security">*[System[(EventID=4732)]]</Select>
          <Select Path="Security">*[System[(EventID=4728)]]</Select>
          <Select Path="Security">*[System[(EventID=4756)]]</Select>
          <Select Path="Security">*[System[(EventID=4733)]]</Select>
          <Select Path="Security">*[System[(EventID=4729)]]</Select>
          <Select Path="Security">*[System[(EventID=4769)]]</Select>
          <Select Path="Security">*[System[(EventID=4768)]]</Select>
          <Select Path="Security">*[System[(EventID=4757)]]</Select>
          <Select Path="Security">*[System[(EventID=5141)]]</Select>
          <Select Path="Security">*[System[(EventID=5137)]]</Select>
          <Select Path="Security">*[System[(EventID=4763)]]</Select>
          <Select Path="Security">*[System[(EventID=4749)]]</Select>
          <Select Path="Security">*[System[(EventID=4753)]]</Select>
          <Select Path="Security">*[System[(EventID=5139)]]</Select>
          <Select Path="Security">*[System[(EventID=5137)]]</Select>
          <Select Path="Security">*[System[(EventID=5136)]]</Select>
          </Query>
          </QueryList>
          </QueryXML>
      </Input>
      
      <Extension multiline>
          Module          xm_multiline
          HeaderLine      /^\d+\/\d\d\/\d\d\d\d+\s\d+:\d\d:\d\d+\s+[A-Z]+/
      </Extension>
      
      <Input filein>
          Module          im_file
          File            'C:\DNSLogs\MGTDCP03_DNS.log'
      #   SavePos         TRUE
          ReadFromLast    FALSE
          InputType       multiline
          <Exec>
           if $raw_event =~ %NOT_STARTING_WITH_DATE_REGEX% drop();
           if $raw_event =~ %EMPTY_EVENT_REGEX% drop();
           if $raw_event =~ %DROP_DNSQUERYEVENT_REGEX% drop();
           if $raw_event =~ %DIRECTION% drop();
           if not ($raw_event =~ %PUBLICIPV4%) drop();
          </Exec>
      </Input>
      
      <Output gelf-1>
          Module om_tcp
          Host 172.17.1.87
          Port 5046
          OutputType  GELF_TCP
      
          <Exec>
            # These fields are needed for Graylog
           # $gl2_source_collector = '${sidecar.nodeId}';
           # $collector_node_id = '${sidecar.nodeName}';
          </Exec>
      
      </Output>
      
      <Output gelf-2>
          Module om_tcp
          Host 172.17.1.87
          Port 5044
          OutputType  GELF_TCP
      
          <Exec>
            # These fields are needed for Graylog.
           # $gl2_source_collector = '${sidecar.nodeId}';
           # $collector_node_id = '${sidecar.nodeName}';
          </Exec>
      
      </Output>
      
      <Route route-1>
        Path filein, eventlog => process-buffer => gelf-1, gelf-2
      </Route>
      

  • manuel.munoz's picture
    (NXLog)

    I would say you need to separate the routes...

    <Route route-1>
      Path filein => process-buffer-1 => gelf-1
      Path eventlog => process-buffer-2 => gelf-2
    </Route>
    

    Also you may need to remove the empty Exec blocks...

        <Exec>
          # These fields are needed for Graylog
         # $gl2_source_collector = '${sidecar.nodeId}';
         # $collector_node_id = '${sidecar.nodeName}';
        </Exec>