Event Logs / Json / Unable to parse at the other end
While not systematic, nxlogs often sends partial json strings over its tcp output. The receiving end is unable to parse it.
The client configuration looks like this:
<Extension json> Module xm_json </Extension> <Input eventlogs_json> Module im_msvistalog Exec delete($Keywords); Exec delete($EventReceivedTime); Exec delete($SourceModuleName); Exec delete($SourceModuleType); Exec $Environment = "development"; Exec to_json(); </Input>
Here's an example truncated log. There are 2 opening brackets but only 1 closed. And there's a weird \r in there too at the end.
"{\"EventTime\":\"2017-04-06 13:39:31\",\"Hostname\":\"redacted\",\"EventType\":\"INFO\",\"SeverityValue\":2,\"Severity\":\"INFO\",\"EventID\":40962,\"SourceName\":\"Microsoft-Windows-PowerShell\",\"ProviderGuid\":\"{A0C1853B-5C40-4B15-8766-3CF1C58F985A}\",\"Version\":1,\"Task\":4,\"OpcodeValue\":2,\"RecordNumber\":1206682,\"ActivityID\":\"{792C42B9-9C21-0001-F1DE-3079219CD201}\",\"ProcessID\":15728,\"ThreadID\":30916,\"Channel\":\"Microsoft-Windows-PowerShell/Operational\",\"Domain\":{\"EventTime\":\"2017-04-06 13:39:30\",\"Hostname\":\"redacted\",\"EventType\":\"INFO\",\"SeverityValue\":2,\"Severity\":\"INFO\",\"EventID\":400,\"SourceName\":\"PowerShell\",\"Task\":4,\"RecordNumber\":2672,\"ProcessID\":0,\"ThreadID\":0,\"Channel\":\"Windows PowerShell\",\"Message\":\"Engine state is changed from None to Available. \\r\\n\\r\\nDetails: \\r\\n\\tNewEngineState=Available\\r\\n\\tPreviousEngineState=None\\r\\n\\r\\n\\tSequenceNumber=13\\r\\n\\r\\n\\tHostName=ConsoleHost\\r\\n\\tHostVersion=4.0\\r\\n\\tHostId=6fc80cc3-85e0-4c9a-a437-1c05ddec479d\\r\\n\\tHostApplication=powershell.exe -command ($(Invoke-WebRequest http://127.0.0.1:redacted/fullstatus -UseBasicParsing).Content | ConvertFrom-Json).\\\"Status\\\"\\r\\n\\tEngineVersion=4.0\\r\\n\\tRunspaceId=cf67d6e2-814b-4d76-82cc-b138ceb2e122\\r\\n\\tPipelineId=\\r\\n\\tCommandName=\\r\\n\\tCommandType=\\r\\n\\tScriptName=\\r\\n\\tCommandPath=\\r\\n\\tCommandLine=\",\"Category\":\"Engine Lifecycle\",\"Opcode\":\"Info\",\"Environment\":\"development\"}\r"
I was able to parse it in python after removeing \r and adding an extra }.
The above JSON is escaped so it's not something that comes straight out of to_json() over TCP. You are not telling the full story.