Splunk Message parsing

Post a different question
3
responses

Hello,

After looking on the community forum i didn't really get the answer i was seeking.

I'm sending windows log to a syslog and to a splunk,

I got some issue about parsing the Message part as you can see : https://cdn.discordapp.com/attachments/700242491227635714/766300884971159562/unknown.png

This is my conf : https://cdn.discordapp.com/attachments/700242491227635714/766301478897451048/unknown.png
https://cdn.discordapp.com/attachments/700242491227635714/766301597541335060/unknown.png

Maybe this is not the good way to do it, i have to send in syslog because i'm sending it after to an elasticsearch and the splunk.

Thank you for your help :)

AskedOctober 15, 2020 - 4:10pm

Answer (1)

Hi,

It would be easier to debug your config if you pasted it as a text - the forum supports markdown. Config file as a two-part screen isn't convenient to analyze ;)

Also, I'm not sure what's the problem? You've posted a screen, but letting know what have you expected would help with suggesting a solution. In other words - what's the question?

Best,

Rafal

AnsweredOctober 19, 2020 - 10:48pm

Comments (2)

  • MB_318874's picture

    Hi Raf,

    Thank you for your time and answer.
    The goal is to parse everything inside the "Messages"

    This is my conf

    ## Please set the ROOT to the folder your nxlog was installed into,
    ## otherwise it will not start.
    define ROOT C:\Program Files (x86)\nxlog
    define ROOT_STRING C:\\Program Files (x86)\\nxlog
    define CERTDIR %ROOT\\cert
    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data
    LogFile %ROOT%\data\nxlog.log

    <Extension _syslog>
    Module xm_syslog
    </Extension>

    <Extension _json>
    Module xm_json
    </Extension>

    <Input internal>
    Module im_internal
    Exec $Message = to_json();
    </Input>

    <Input eventlog>
    Module im_msvistalog

    Query <QueryList>\
    <Query Id="0">\
    <Select Path="Security">*</Select>\
    <Select Path="Application">*</Select>\
    <Select Path="System">*</Select>\
    <Select Path="Windows Powershell">*</Select>\
    <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>\
    </Query>\
    </QueryList>

    Exec $Message = to_json();

    Exec if ($Application =~ /nxlog\\nxlog.exe/) drop();

    Exec if ($CommandLine =~ /"C:\\Program Files \(x86\)\\nxlog\\nxlog.exe" -c "C:\\Program Files \(x86\)\\nxlog\\conf\\nxlog.conf"/) drop();

    </Input>

    <Output out>
    Module om_tcp
    Host syslogIP(i change it ;) )
    Port 514

    Exec to_syslog_ietf(); $raw_event = replace($raw_event, 'NXLOG@12345', '4e7d9816-e12f-3cfe-bd84-5d9042eb2c97@41058 tag="windows"] [', 1);
    </Output>

    <Route 1>
    Path internal, eventlog => out
    </Route>

    #######
    This my RAW log after i keep only the json part
    {"EventTime":"2020-10-21 01:15:12","Hostname":"wks01.domain.local","Keywords":36028797018963968,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":800,"SourceName":"PowerShell","Task":8,"RecordNumber":3911888,"ProcessID":0,"ThreadID":0,"Channel":"Windows PowerShell","Message":"Pipeline execution details for command line: dir. \r\n\r\nContext Information: \r\n\tDetailSequence=1\r\n\tDetailTotal=1\r\n\r\n\tSequenceNumber=21\r\n\r\n\tUserId=domain\\username\r\n\tHostName=ConsoleHost\r\n\tHostVersion=5.1.18362.145\r\n\tHostId=1cfb6616-f938-4221-9970-b204702e8c61\r\n\tHostApplication=C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe\r\n\tEngineVersion=5.1.18362.145\r\n\tRunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d\r\n\tPipelineId=6\r\n\tScriptName=\r\n\tCommandLine=dir \r\n\r\nDetails: \r\nCommandInvocation(Get-ChildItem): \"Get-ChildItem\"\r\n","Category":"Pipeline Execution Details","Opcode":"Info","EventReceivedTime":"2020-10-21 01:15:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"}

    And this is how i see it.

    { [-]
    Category: Pipeline Execution Details
    Channel: Windows PowerShell
    EventID: 800
    EventReceivedTime: 2020-10-21 01:15:13
    EventTime: 2020-10-21 01:15:12
    EventType: INFO
    Hostname: wks01.domain.local
    Keywords: 36028797018963970
    Message: Pipeline execution details for command line: dir.

    Context Information:
    DetailSequence=1
    DetailTotal=1

    SequenceNumber=21

    UserId=domain\username
    HostName=ConsoleHost
    HostVersion=5.1.18362.145
    HostId=1cfb6616-f938-4221-9970-b204702e8c61
    HostApplication=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    EngineVersion=5.1.18362.145
    RunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d
    PipelineId=6
    ScriptName=
    CommandLine=dir

    Details:
    CommandInvocation(Get-ChildItem): "Get-ChildItem"

    Opcode: Info
    ProcessID: 0
    RecordNumber: 3911888
    Severity: INFO
    SeverityValue: 2
    SourceModuleName: eventlog
    SourceModuleType: im_msvistalog
    SourceName: PowerShell
    Task: 8
    ThreadID: 0
    }

    My question is, can we have a configuration where the things inside the Message field already parse by the nxlog?

    This part :

    Message: Pipeline execution details for command line: dir.

    Context Information:
    DetailSequence=1
    DetailTotal=1

    SequenceNumber=21

    UserId=domain\username
    HostName=ConsoleHost
    HostVersion=5.1.18362.145
    HostId=1cfb6616-f938-4221-9970-b204702e8c61
    HostApplication=C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    EngineVersion=5.1.18362.145
    RunspaceId=c9a23581-4e32-4eba-8b7e-778a7cfd541d
    PipelineId=6
    ScriptName=
    CommandLine=dir

    Details:
    CommandInvocation(Get-ChildItem): "Get-ChildItem"

    I have find some tips on the forum but i can't get a result :/... Windows XML log :'(

    Thank you for your time and help!

    October 21, 2020 - 10:23am