CSV file in Linux

Tags: LINUX | csv

#1 multiplierx

Hello everyone,



As you can see from my NXlog config below, I am pulling the Linux auth logs in and passing them to our SIEM platform. The auth logs work fine, however as you can also see I am trying to pull in a CSV file that is located on the Linux box. Each individual event is surrounded by curly brackets e.g: {'event}, so I would like to just pull each event into a field called message and then parse these messages on our SIEM platform. I can't seem to get this to work whatsoever, the TCP connection initiates, but NXlog doesn't pickup the CSV file. Any ideas?



 



Cheers



G




########################################

# Global directives                    #

########################################

User nxlog

Group nxlog

LogFile /var/log/nxlog/nxlog.log

LogLevel INFO

########################################

# Modules                              #

########################################

<Extension _syslog>

Module  xm_syslog

</Extension>



<Input auth_logs>

        Module  im_file

        File    "/var/log/auth.log"

        SavePos TRUE

        ReadFromLast    TRUE

</Input>



<Output to_relay>

        Module  om_tcp

        Host    127.0.0.1

        Port    20009

        OutputType      LineBased

</Output>



########################################

# Routes                               #

########################################

<Route 1>

        Path    auth_logs => to_relay

</Route>



<Extension csv1>

        Module  xm_csv

        Fields  $Message

        Delimiter       '{'

</Extension>



<Input filein>

        Module  im_file

        File    "/etc/ingest/sucuri/sucuri.csv"

        Exec    csv1->parse_csv();

</Input>



<Output test>

        Module  om_tcp

        Host    127.0.0.1

        Port    20002

        OutputType      Binary

</Output>



<Route 2>

        Path    filein => test

</Route>



#2 b0ti Nxlog ✓
#1 multiplierx
Hello everyone, As you can see from my NXlog config below, I am pulling the Linux auth logs in and passing them to our SIEM platform. The auth logs work fine, however as you can also see I am trying to pull in a CSV file that is located on the Linux box. Each individual event is surrounded by curly brackets e.g: {'event}, so I would like to just pull each event into a field called message and then parse these messages on our SIEM platform. I can't seem to get this to work whatsoever, the TCP connection initiates, but NXlog doesn't pickup the CSV file. Any ideas?   Cheers G ######################################## # Global directives                    # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules                              # ######################################## <Extension _syslog> Module  xm_syslog </Extension> <Input auth_logs>         Module  im_file         File    "/var/log/auth.log"         SavePos TRUE         ReadFromLast    TRUE </Input> <Output to_relay>         Module  om_tcp         Host    127.0.0.1         Port    20009         OutputType      LineBased </Output> ######################################## # Routes                               # ######################################## <Route 1>         Path    auth_logs => to_relay </Route> <Extension csv1>         Module  xm_csv         Fields  $Message         Delimiter       '{' </Extension> <Input filein>         Module  im_file         File    "/etc/ingest/sucuri/sucuri.csv"         Exec    csv1->parse_csv(); </Input> <Output test>         Module  om_tcp         Host    127.0.0.1         Port    20002         OutputType      Binary </Output> <Route 2>         Path    filein => test </Route>

I doubt that the delimiter is `{` , this sounds rather like JSON to me.

Unfortunately without a sample it is hard to tell and give advice.