5
responses

The article on structured logging (https://nxlog.co/why-use-structured-logging) shows how you should use structured logging so that changes in log format is minimized.  The example of the sysmon event, process creates, shows what I think is a bug in NXLog.

 

The ProcessID in the "Message" is the ProcessID (25848) of the new process that sysmon sees created.  However, in the structured NXLog key/values, the ProcessID is that of sysmon itself (1680)

 

The only way to get the ProcessID of the process sysmon observed created, is to use regular expressions.  Can you verify that this is a bug in NXLog?

AskedMay 2, 2016 - 3:22pm

Answer (1)

This is a known issue. Unfortunately in the case of sysmon logs there is a ProcessID field under EventData while within the System properties there is also a ProcessID so the former does not get captured. See this question.

Comments (4)

  • cybergoof's picture

    But, in sysmon, the desired ProcessID is in the <EventData> key.  In the JSON created by NXLOG, if they were created under a "EventData":{} object.  This would solve the namespace collision problem.

  • adm's picture
    (NXLog)

    NXLog stores the data internally in a flat list of key-value pairs, not JSON. Nested values are not supported (i.e. such structure needs to be flattened). Having support for such complex structure would have some benefits and would also bring some more problems with the added complexity.

  • cybergoof's picture

    Thanks for the response.  I would suggest making this a future feature.  It seems elastic.co's new BEATS log forwarder handles the sysmon ProcessID correctly.  

  • IHK-Dresden's picture

    Is it conceivable that to avoid naming conflicts all newly in the message field captured names could get a (configurable) prefix (for example prefix "Message." like "Message.ProcessId" or prefix "EventData." like "EventData.ProcessId")?