How to add a field for the file offset?
We are pushing logs from file with the im_file module to logstash and then to elasticsearch. However, some of these logs only have second accuracy, and therefore not returned in order by elasticsearch when sorted by just the time. To get around this problem, we would like to add the position of the log event to a field that we store (for example beginning line number or byte offset within the source file). Is there a way I can add a separate field for this in the events pushed by nxlog, either in the input or output configuration?
It is not possible to retrieve the file offset or line number but you could get around this by adding a serial number using statistical counters or variables.
Also note that the $EventReceivedTime field is automatically set when the line is read. This contains a millisecond precision value.
