responses
Hello,
After looking on the community forum i didn't really get the answer i was seeking.
I'm sending windows log to a syslog and to a splunk,
I got some issue about parsing the Message part as you can see : https://cdn.discordapp.com/attachments/700242491227635714/766300884971159562/unknown.png
This is my conf : https://cdn.discordapp.com/attachments/700242491227635714/766301478897451048/unknown.png
https://cdn.discordapp.com/attachments/700242491227635714/766301597541335060/unknown.png
Maybe this is not the good way to do it, i have to send in syslog because i'm sending it after to an elasticsearch and the splunk.
Thank you for your help :)
Comments (10)
Hi Raf,
Thank you for your time and answer.
The goal is to parse everything inside the "Messages"
This is my conf
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
define ROOT C:\Program Files (x86)\nxlog
define ROOT_STRING C:\\Program Files (x86)\\nxlog
define CERTDIR %ROOT\\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Input internal>
Module im_internal
Exec $Message = to_json();
</Input>
<Input eventlog>
Module im_msvistalog
Query <QueryList>\
<Query Id="0">\
<Select Path="Security">*</Select>\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Windows Powershell">*</Select>\
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>\
</Query>\
</QueryList>
Exec $Message = to_json();
Exec if ($Application =~ /nxlog\\nxlog.exe/) drop();
Exec if ($CommandLine =~ /"C:\\Program Files \(x86\)\\nxlog\\nxlog.exe" -c "C:\\Program Files \(x86\)\\nxlog\\conf\\nxlog.conf"/) drop();
</Input>
<Output out>
Module om_tcp
Host syslogIP(i change it ;) )
Port 514
Exec to_syslog_ietf(); $raw_event = replace($raw_event, 'NXLOG@12345', '4e7d9816-e12f-3cfe-bd84-5d9042eb2c97@41058 tag="windows"] [', 1);
</Output>
<Route 1>
Path internal, eventlog => out
</Route>
#######
This my RAW log after i keep only the json part
{"EventTime":"2020-10-21 01:15:12","Hostname":"wks01.domain.local","Keywords":36028797018963968,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":800,"SourceName":"PowerShell","Task":8,"RecordNumber":3911888,"ProcessID":0,"ThreadID":0,"Channel":"Windows PowerShell","Message":"Pipeline execution details for command line: dir. \r\n\r\nContext Information: \r\n\tDetailSequence=1\r\n\tDetailTotal=1\r\n\r\n\tSequenceNumber=21\r\n\r\n\tUserId=domain\\username\r\n\tHostName=ConsoleHost\r\n\tHostVersion=5.1.18362.145\r\n\tHostId=1cfb6616-f938-4221-9970-b204702e8c61\r\n\tHostApplication=C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe\r\n\tEngineVersion=5.1.18362.145\r\n\tRunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d\r\n\tPipelineId=6\r\n\tScriptName=\r\n\tCommandLine=dir \r\n\r\nDetails: \r\nCommandInvocation(Get-ChildItem): \"Get-ChildItem\"\r\n","Category":"Pipeline Execution Details","Opcode":"Info","EventReceivedTime":"2020-10-21 01:15:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"}
And this is how i see it.
{ [-]
Category: Pipeline Execution Details
Channel: Windows PowerShell
EventID: 800
EventReceivedTime: 2020-10-21 01:15:13
EventTime: 2020-10-21 01:15:12
EventType: INFO
Hostname: wks01.domain.local
Keywords: 36028797018963970
Message: Pipeline execution details for command line: dir.
Context Information:
DetailSequence=1
DetailTotal=1
SequenceNumber=21
UserId=domain\username
HostName=ConsoleHost
HostVersion=5.1.18362.145
HostId=1cfb6616-f938-4221-9970-b204702e8c61
HostApplication=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
EngineVersion=5.1.18362.145
RunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d
PipelineId=6
ScriptName=
CommandLine=dir
Details:
CommandInvocation(Get-ChildItem): "Get-ChildItem"
Opcode: Info
ProcessID: 0
RecordNumber: 3911888
Severity: INFO
SeverityValue: 2
SourceModuleName: eventlog
SourceModuleType: im_msvistalog
SourceName: PowerShell
Task: 8
ThreadID: 0
}
My question is, can we have a configuration where the things inside the Message field already parse by the nxlog?
This part :
Message: Pipeline execution details for command line: dir.
Context Information:
DetailSequence=1
DetailTotal=1
SequenceNumber=21
UserId=domain\username
HostName=ConsoleHost
HostVersion=5.1.18362.145
HostId=1cfb6616-f938-4221-9970-b204702e8c61
HostApplication=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
EngineVersion=5.1.18362.145
RunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d
PipelineId=6
ScriptName=
CommandLine=dir
Details:
CommandInvocation(Get-ChildItem): "Get-ChildItem"
I have find some tips on the forum but i can't get a result :/... Windows XML log :'(
Thank you for your time and help!
I think this is what you are looking for: https://nxlog.co/documentation/nxlog-user-guide/powershell-activity.html
Ok thank, it try but nothing change
is my conf right?
le nxlog log file tel me he "couldn't parse statement at line 73, character 57 in C....nx.conf; procedure 'parse_kvp()' does not exist or takes different arguments
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
define ROOT C:\Program Files (x86)\nxlog
define ROOT_STRING C:\\Program Files (x86)\\nxlog
define CERTDIR %ROOT\\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Input internal>
Module im_internal
Exec $Message = to_json();
</Input>
<Input eventlog>
Module im_msvistalog
Query <QueryList>\
<Query Id="0">\
<Select Path="Security">*</Select>\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Windows Powershell">*</Select>\
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>\
</Query>\
</QueryList>
Exec $Message = to_json();
Exec if ($Application =~ /nxlog\\nxlog.exe/) drop();
Exec if ($CommandLine =~ /"C:\\Program Files \(x86\)\\nxlog\\nxlog.exe" -c "C:\\Program Files \(x86\)\\nxlog\\conf\\nxlog.conf"/) drop();
</Input>
<Extension kvp>
Module xm_kvp
KVPDelimiter ,
KVDelimiter =
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0" Path="Microsoft-Windows-PowerShell/Operational">
<Select Path="Microsoft-Windows-PowerShell/Operational">
*[System[EventID=4103]]</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if defined($ContextInfo)
{
$ContextInfo = replace($ContextInfo, "\r\n", ",");
$ContextInfo = replace($ContextInfo, "\n", ",");
$ContextInfo = replace($ContextInfo, " ", "");
kvp->parse_kvp($ContextInfo, "ContextInfo_");
delete($ContextInfo);
delete($Message);
}
json->to_json();
</Exec>
</Input>
<Output out>
Module om_tcp
Host syslogIP(i change it ;) )
Port 514
Exec to_syslog_ietf(); $raw_event = replace($raw_event, 'NXLOG@12345', '4e7d9816-e12f-3cfe-bd84-5d9042eb2c97@41058 tag="windows"] [', 1);
</Output>
<Route 1>
Path internal, eventlog, in => out
</Route>
Hello,
Any idea? :/ I'm stuck....
Hello,
first - the error messages themselves give us some clues about what's going on. Here, it's stated
does not exist or takes different arguments
.The error comes from
parse_kvp()
method, and can be found in the reference manual. Here, we can see the whole configuration of the module.In your case, simply remove the method's arguments, so change
to
You have at least one more error - the syntax used by you is inappropriate - instead of
json->to_json();
you need use justto_json();
.Good luck!
Rafal
Thank you Raf!! it's working better :D
So i guess your powershell documentation is wrong, because i take it from there..?
Because i can't parse what is have inside the Message part i guess because we remove the match of kvp->parse_kvp($ContextInfo, "ContextInfo_"); no ?
Could you point me to the exact place where did you take it from?
Thanks,
Rafal
Hi Rafal,
first : thank you for your time and answer :)
Sure : https://nxlog.co/documentation/nxlog-user-guide/powershell-activity.html
In this guide they have the output like that :
"ContextInfo_Severity": "Informational",
"ContextInfo_Host Name": "ConsoleHost",
"ContextInfo_Host Version": "5.1.17763.592",
"ContextInfo_Host ID": "67d049eb-f3d6-4718-8cd2-b9dae30c4c7b",
"ContextInfo_Host Application": "C: \\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
"ContextInfo_Engine Version": "5.1.17763.592",
"ContextInfo_Runspace ID": "3145a9e1-18e3-4fa1-8700-fc78c783684b",
but because we remove the line where we have kvp->parse_kvp($ContextInfo, "ContextInfo_");
I don't have the output of this document. Maybe you need to update it?
Other topics on the powershell too. I use your documentation to configure the transcript powershell but when i use the configuration from this chapter : 113.2.3.
The configuration file : https://gitlab.com/nxlog-public/contrib/raw/master/guide_configs/powershell_transcription-nxlog.conf?inline=false
I got this when i start the nxlog agent
2020-11-24 21:19:02 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:140; couldn't parse statement at line 152, character 16 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; invalid character: '$' (0x24)
2020-11-24 21:19:02 ERROR module 'transcription' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:194
2020-11-24 21:19:02 INFO connecting to 192.168.134.20:514
$ more C:\Users\Public\log.txt
0-11-24 21:18:41 WARNING stopping nxlog service
2020-11-24 21:18:41 WARNING nxlog-ce received a termination request signal, exiting...
2020-11-24 21:19:02 WARNING not starting unused module transcription
2020-11-24 21:19:02 INFO nxlog-ce-2.10.2102 started
Thank you for your answer
I don't like resurrecting dead posts, but I'm going to do it anyway. Did you ever figure this out? I'm having the same issue and unable to find a solution. The documentation is still wrong?
https://nxlog.co/documentation/nxlog-user-guide/powershell-activity.html