NXLog postgresql ID manipulation?

Tags:

#22 JaVa
#21 b0ti Nxlog ✓
What's wrong with what was suggested before? Exec $Message = to_kvp(); to_syslog_bsd(); If you see the data written to the file it should also work with your syslog server.

It should work but it doesn't. Now as i edited the configuration from this:

<Route 1> Path dbi => file,out </Route>

To this:

<Route 1> Path dbi => out </Route>

I got a TCP connection to the SIEM system in the receiving end but again no data is transfered. Apparently that Route module doesn't support multiple output paths?

Also i deleted the configcache.dat -file and NXLog does not create it and still remembers the last read ID from the database.

2019-02-01 13:57:40 DEBUG no entries found, not writing configcache.dat 2019-02-01 13:57:40 DEBUG nxlog_shutdown() leave 2019-02-01 13:57:40 DEBUG reading config cache from /opt/nxlog/var/spool/nxlog/configcache.dat

File list at that time:

ls -la /opt/nxlog/var/spool/nxlog/ total 8 drwxrwx---. 2 nxlog nxlog 4096 Feb 1 13:56 . drwxrwxr-x. 3 root root 4096 Jan 24 09:59 ..

The logs are like this now when i try to send the logs to remote:

2019-02-01 13:57:40 DEBUG RESUME: dbi 2019-02-01 13:57:40 DEBUG module dbi already running, skipping resume 2019-02-01 13:57:40 DEBUG worker 0 processing event 0x7f7acc009110 2019-02-01 13:57:40 DEBUG PROCESS_EVENT: READ (dbi) 2019-02-01 13:57:40 DEBUG im_dbi sql: SELECT id, discriminator, time, queryid, message, timestamprecord, response, memberclass, membercode, subsystemcode FROM logrecord WHERE id > -1 LIMIT 50 2019-02-01 13:57:40 DEBUG worker 2 got signal for new job 2019-02-01 13:57:40 DEBUG worker 2 got no event to process 2019-02-01 13:57:40 DEBUG worker 2 waiting for new event 2019-02-01 13:57:40 DEBUG im_dbi read 50 rows 2019-02-01 13:57:40 DEBUG executing statements 2019-02-01 13:57:40 DEBUG before nx_logqueue_push, size: 0 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG event added to jobqueue 2019-02-01 13:57:40 DEBUG executing statements 2019-02-01 13:57:40 DEBUG before nx_logqueue_push, size: 1 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG executing statements 2019-02-01 13:57:40 DEBUG before nx_logqueue_push, size: 2 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG executing statements

2019-02-01 13:57:40 DEBUG event added to jobqueue 2019-02-01 13:57:40 DEBUG worker 1 processing event 0x7f7acc0176c0 2019-02-01 13:57:40 DEBUG PROCESS_EVENT: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG om_tcp_write 2019-02-01 13:57:40 DEBUG add socket [21] 2019-02-01 13:57:40 DEBUG socket already added to pollset with reqevents [21 != 21] 2019-02-01 13:57:40 DEBUG out get_next_logdata: got (queuesize: 25) 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG event added to jobqueue 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: MODULE_RESUME (dbi) 2019-02-01 13:57:40 DEBUG event added to jobqueue 2019-02-01 13:57:40 DEBUG executing statements 2019-02-01 13:57:40 DEBUG worker 2 got signal for new job 2019-02-01 13:57:40 DEBUG worker 2 got no event to process 2019-02-01 13:57:40 DEBUG worker 2 waiting for new event 2019-02-01 13:57:40 DEBUG worker 3 got signal for new job 2019-02-01 13:57:40 DEBUG worker 3 got no event to process 2019-02-01 13:57:40 DEBUG worker 3 waiting for new event 2019-02-01 13:57:40 DEBUG before nx_logqueue_push, size: 26 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out) 2019-02-01 13:57:40 DEBUG executing statements 2019-02-01 13:57:40 DEBUG before nx_logqueue_push, size: 27 2019-02-01 13:57:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (out)

And the next query after that WHERE id > -1 is this:

2019-02-01 13:57:40 DEBUG im_dbi sql: SELECT id, discriminator, time, queryid, message, timestamprecord, response, memberclass, membercode, subsystemcode FROM logrecord WHERE id > 761 LIMIT 50 2019-02-01 13:57:40 DEBUG im_dbi read 29 rows

And after that its the end of the database:

2019-02-01 13:57:40 DEBUG im_dbi sql: SELECT id, discriminator, time, queryid, message, timestamprecord, response, memberclass, membercode, subsystemcode FROM logrecord WHERE id > 789 LIMIT 50 2019-02-01 13:57:40 DEBUG im_dbi read 3 rows