Need help ingesting logs from two different sources - Page 1

#1 aleblanc75 2 years ago

I've been trying to figure out the best way to ingest logs from 2 different data streams and have them go to separate log files.

Heres the copy of my configuration. For my first input i have a bunch of firewall logs coming in to /syslog/firewalls.log. I now want to ingest syslog data from my isilon to a different log file. It only seems to work if I have host 0.0.0.0 setup. I'm getting the data but everything is being written to firewalls.log and not my isilon.log

Any help would be greatly appreciated.

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension _json>
Module xm_json
</Extension>

<Input udp1>
Module im_udp
Host 0.0.0.0
Port 514
Exec parse_syslog();
</Input>

<Output fwlog>
Module om_file
File "/syslog/firewalls.log"
Exec to_json();
</Output>

<Input udp2>
Module im_udp
Host 0.0.0.0
Port 514
InputType Syslog_TLS
Exec parse_syslog();
</Input>

<Output isilog>
Module om_file
File "/syslog/isilon.log"
Exec to_json();
</Output>

########################################
# Routes #
########################################

<Route udp_to_file1>
Path udp1 => fwlog
</Route>

<Route udp_to_file2>
Path udp2 => isilog
</Route>

Permalink

#2 jeffron Nxlog ✓ 2 years ago

#1 aleblanc75

I've been trying to figure out the best way to ingest logs from 2 different data streams and have them go to separate log files. Heres the copy of my configuration. For my first input i have a bunch of firewall logs coming in to /syslog/firewalls.log. I now want to ingest syslog data from my isilon to a different log file. It only seems to work if I have host 0.0.0.0 setup. I'm getting the data but everything is being written to firewalls.log and not my isilon.log Any help would be greatly appreciated. <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input udp1> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); </Input> <Output fwlog> Module om_file File "/syslog/firewalls.log" Exec to_json(); </Output> <Input udp2> Module im_udp Host 0.0.0.0 Port 514 InputType Syslog_TLS Exec parse_syslog(); </Input> <Output isilog> Module om_file File "/syslog/isilon.log" Exec to_json(); </Output> ######################################## # Routes # ######################################## <Route udp_to_file1> Path udp1 => fwlog </Route> <Route udp_to_file2> Path udp2 => isilog </Route>

Hi Adam,

it appears you are collecting logs from the same source 0.0.0.0:514. is that is the case, you can send it to two different outputs, with one route, thus eliminating the need for udp2 thus.

<Route udp_to_file1>
    Path udp1 => fwlog, isilog
</Route>

Alternatively, if you are collecting logs from different sources, please specify the correct port for the second input. eg 0.0.0.0:515

I hope this helps.

Regards,

Jeffron