NXlog to read new log and send content of file to syslog server

Post a different question
8
responses

Hello Team,

I am new to Nxlog and we have a requirement to send below log file content to syslog server, fields will be same for every new log file.
Can you please help in writing conf file to send it to syslog.

Sample Log file.

OPSWAT - METADEFENDER KIOSK SCAN RESULTS

User ID: TRAININGLAB\syslog
Profile: Default
Session ID: 52CE90C9-73DC-4150-AE7F-1FDCFF933D3F

PROCESSING FINISHED SUCCESSFULLY

Process Start Time: 2019-09-21 16:09:36
Process Finish Time: 2019-09-21 16:10:55

MetaDefender Kiosk Version: 4.3.5.2010
MetaDefender Core Version: 4.16.2

Device Information
Manufacturer: (Standard disk drives)
Model: SanDisk Cruzer Blade USB Device
Serial Number: 4C530000260530107000
Device ID: USBSTOR\DISK&VEN_SANDISK&PROD_CRUZER_BLADE&REV_1.00\4C530000260530107000&0
Media Type: USB Device
Partition Count: 1
Partition Name: Disk #1, Partition #0
- Bootable: NO
Disk Usage: 39MB / 14GB
Scanning System: SYSTEM1

Full Media Scanned: NO
Full Media Processed: NO
Total Files Scanned: 7
Total Files Processed: 7

Blocked Files
No blocked files found

Blocked Actions Taken
- Sanitized: 0
- Quarantined: 0
- Deleted: 0
- Post Action Ran: 0
- Copied To Media: 0
- Copied To Directory: 0
- Copied To Vault Server: 0
- Moved To Media: 0
- Moved To Directory: 0
- Moved To Vault Server: 0
- Destination Media Wiped: NO

Allowed Actions Taken
- Sanitized: 0
- Post Action Ran: 0
- Copied To Media: 0
- Copied To Directory: 0
- Copied To Vault Server: 0
- Moved To Media: 0
- Moved To Directory: 0
- Moved To Vault Server: 0
- Destination Media Wiped: NO

Skipped Files: 0

Failed To Delete: 0

File Type Totals
application/octet-stream: 1
application/pdf: 2
application/x-dosexec: 2
text/plain: 2

_______________________________________________________________________

ALLOWED FILES

1. PATH: E:\New Text Document (6).txt
SHA-256: E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855
-----------------------------------------------------------------------
2. PATH: E:\SanDiskSecureAccess\DownloadSanDiskSecureAccess_Mac.pdf
SHA-256: B2E01B65F369095428DB35D59A41FDB80A5B16F6C496D7420D814B63CC8EEDCB
-----------------------------------------------------------------------
3. PATH: E:\SanDiskSecureAccess\SanDisk_SecureAccess_QSG.PDF
SHA-256: BC6D908229CA23F0FA78690BF5CD498F67A6FDB5CD368A4F89BABC98427A93CB
-----------------------------------------------------------------------
4. PATH: E:\New Text Document (7).txt
SHA-256: BDED243D2EEDEEF19D62D88A361A7019A007363BBBF429A873320015B865A456
-----------------------------------------------------------------------
5. PATH: E:\sgbox.txt
SHA-256: CA3ED41768F78C7E61BC782716010A208DE09464BDEB283D27DABE57318EE3B8
-----------------------------------------------------------------------
6. PATH: E:\SanDiskSecureAccessV3.1_win.exe
SHA-256: 829F3BC240D26077AC00CE58B15D7D349E5D473B83629D3CC404A34BA865C9EC
-----------------------------------------------------------------------
7. PATH: E:\epm.exe
SHA-256: 3D878E578E7340443785D4DC6CEA0A5B415D3BB107AFB0282DFEBF776930B216
-----------------------------------------------------------------------

_______________________________________________________________________

AskedOctober 6, 2019 - 8:31pm

Comments (6)

  • sunilj777's picture

    Nxlog.conf which I have used

    Panic Soft
    #NoFreeOnExit TRUE

    define ROOT C:\Program Files (x86)\nxlog
    define CERTDIR %ROOT%\cert
    define CONFDIR %ROOT%\conf
    define LOGDIR %ROOT%\data
    define LOGFILE %LOGDIR%\nxlog.log
    LogFile %LOGFILE%

    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data

    <Extension syslog>
    Module xm_syslog

    </Extension>

    <Extension multiline_parser>
    Module xm_multiline
    #HeaderLine "OPSWAT - METADEFENDER KIOSK SCAN RESULTS"
    HeaderLine /.*OPSWAT.*/

    </Extension>

    <Input log_file>
    Module im_file
    # define the file path (es: below)
    # to collect logs from different files you need to indicate the format of the files for example .txt as in the following example.
    File 'c:\Program Files (x86)\OPSWAT\Metadefender Kiosk\Client\Log\*-*-*-*-*.log'
    InputType multiline_parser
    Exec to_syslog_bsd();

    </Input>

    <Output log_file_out>
    Module om_udp
    # ip address/hostname of the SGBox
    Host *************
    # Port
    Port 514

    </Output>

    <Route log_file>
    Path log_file => log_file_out
    </Route>

    October 6, 2019 - 8:41pm
  • sunilj777's picture

    Nxlog is sending the file content as multiline to syslog, but it also sending some information which is not requried.

    Example:

    Not Required:

    OPSWAT - METADEFENDER KIOSK SCAN RESULTS

    PROCESSING FINISHED SUCCESSFULLY

    _______________________________________________________________________

    ----------------------------------------------------------------------

    October 6, 2019 - 8:44pm
  • manuel.munoz's picture

    Hi sunilj777,

    We would suggest you using the following configuration...

    <Extension multiline_parser>
    Module      xm_multiline
    HeaderLine  /.*OPSWAT.*/
    Exec        if $raw_event =~ /^(---|___|PROCESSING FINISHED SUCCESSFULLY).*/ drop();
</Extension>

<Input log_file>
    Module      im_file
    File        '/tmp/user-example.log'
    InputType   multiline_parser
    Exec        if $raw_event =~ s/OPSWAT - METADEFENDER KIOSK SCAN RESULTS\s//;
    Exec        to_syslog_bsd();
</Input>

<Output log_file_out>
    Module      om_file
    File        '/tmp/forum_multi.out'
    Exec        log_info($raw_event);
</Output>

<Route log_file>
    Path        log_file => log_file_out
</Route>

    The results after processing your example file...

    manuel@portatil-ubuntu:~$ more /tmp/forum_multi.out
<13>Oct  8 17:58:05 portatil-ubuntu  User ID: TRAININGLAB\syslog Profile: Default Session ID: 52CE90C9-73DC-4150-AE7F-1FDCFF933D3F   Process Start Time: 2019-09-21 16:09:36 Proces
s Finish Time: 2019-09-21 16:10:55  MetaDefender Kiosk Version: 4.3.5.2010 MetaDefender Core Version: 4.16.2  Device Information Manufacturer: (Standard disk drives) Model: SanDis
k Cruzer Blade USB Device Serial Number: 4C530000260530107000 Device ID: USBSTOR\DISK&VEN_SANDISK&PROD_CRUZER_BLADE&REV_1.00\4C530000260530107000&0 Media Type: USB Device Partitio
n Count: 1 Partition Name: Disk #1, Partition #0 - Bootable: NO Disk Usage: 39MB / 14GB Scanning System: SYSTEM1  Full Media Scanned: NO Full Media Processed: NO Total Files Scann
ed: 7 Total Files Processed: 7  Blocked Files No blocked files found  Blocked Actions Taken - Sanitized: 0 - Quarantined: 0 - Deleted: 0 - Post Action Ran: 0 - Copied To Media: 0 
- Copied To Directory: 0 - Copied To Vault Server: 0 - Moved To Media: 0 - Moved To Directory: 0 - Moved To Vault Server: 0 - Destination Media Wiped: NO  Allowed Actions Taken - 
Sanitized: 0 - Post Action Ran: 0 - Copied To Media: 0 - Copied To Directory: 0 - Copied To Vault Server: 0 - Moved To Media: 0 - Moved To Directory: 0 - Moved To Vault Server: 0 
- Destination Media Wiped: NO  Skipped Files: 0  Failed To Delete: 0  File Type Totals application/octet-stream: 1 application/pdf: 2 application/x-dosexec: 2 text/plain: 2   ALLO
WED FILES  1. PATH: E:\New Text Document (6).txt SHA-256: E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855 2. PATH: E:\SanDiskSecureAccess\DownloadSanDiskSecureAcc
ess_Mac.pdf SHA-256: B2E01B65F369095428DB35D59A41FDB80A5B16F6C496D7420D814B63CC8EEDCB 3. PATH: E:\SanDiskSecureAccess\SanDisk_SecureAccess_QSG.PDF SHA-256: BC6D908229CA23F0FA78690
BF5CD498F67A6FDB5CD368A4F89BABC98427A93CB 4. PATH: E:\New Text Document (7).txt SHA-256: BDED243D2EEDEEF19D62D88A361A7019A007363BBBF429A873320015B865A456 5. PATH: E:\sgbox.txt SHA
-256: CA3ED41768F78C7E61BC782716010A208DE09464BDEB283D27DABE57318EE3B8 6. PATH: E:\SanDiskSecureAccessV3.1_win.exe SHA-256: 829F3BC240D26077AC00CE58B15D7D349E5D473B83629D3CC404A34
BA865C9EC 7. PATH: E:\epm.exe SHA-256: 3D878E578E7340443785D4DC6CEA0A5B415D3BB107AFB0282DFEBF776930B216

    October 8, 2019 - 6:04pm
  • sunilj777's picture

    Hello Team,

    We are looking for few modifications in the result to drop SHA-256 Vaule, the result should only display the file names and ignore SHA-256 value

    Other requirement on the below lines, we are looking if the "Blocked Files result shows "No blocked files found" we want to replace or modify it as Blacklisted: 0 (Only for matching string-No blocked files found) and if the result shows Blocked Files Blacklisted: 4 (example 4) nxlog should ignore it.

    Blocked Files
    No blocked files found - It should be replaced with Blacklisted an number 0
    Out come in Syslog
    Blocked Files
    Blacklisted: 0
    Other Scenario, Ignore if the log file has "No blocked files found" (No changes- send as it to syslog)

    Regards
    Sunil

    February 25, 2020 - 9:48am

Answer (1)

<Extension multiline_parser>
    Module      xm_multiline
    HeaderLine  /.*OPSWAT.*/
    Exec        if $raw_event =~ /^(---|___|PROCESSING FINISHED SUCCESSFULLY).*/ drop();
</Extension>

<Input log_file>
    Module      im_file
    File        '/tmp/user-example.log'
    InputType   multiline_parser
    Exec        if $raw_event =~ s/OPSWAT - METADEFENDER KIOSK SCAN RESULTS\s//;
    Exec        to_syslog_bsd();
</Input>

<Output log_file_out>
    Module      om_file
    File        '/tmp/forum_multi.out'
</Output>

<Route log_file>
    Path        log_file => log_file_out
</Route>
AnsweredOctober 17, 2019 - 12:10pm

Comments (1)

  • sunilj777's picture

    Thanks Manuel for quick reply. I posted this long back and was not expected the answer so quickly.
    Looking at the date I see it was answered in just few days and I was really very happy to see the reply so fast. I didn't imagine it usually the community reply's will be late and very late.

    Thanks again for the quick reply
    Regards
    Sunil Prabhakar

    January 15, 2020 - 7:12pm