4
responses

I'm attempting to demo xnlog and running into a problem where the Windows Server 2016 event logs are being sent to AWS ElasticSearch Service with the EventTime being a string. This basically renders it impossible to index the logs, as the Kibana board requires a time-field name and is not recongizing the string as a datetime.  Any suggestions on this, or is this a potential bug with Server 2016?

AskedJanuary 6, 2017 - 7:08pm

Answer (1)

It's very unlikely that this is due to Windows 2016. Perhaps there is something in your config that converts it to a string but it's hard to tell the cause without that.

Comments (3)

  • chris.bowen's picture

    Here's my current configuration:

    <Extension syslog>
        Module      xm_syslog
    </Extension>
    
    <Extension json>
        Module      xm_json
    </Extension>
    
    <Input EventIn>
        Module im_msvistalog
        ReadFromLast TRUE
        <QueryXML>
            <QueryList>
                <Query Id='1'>
                    <Select Path='Application'>*</Select>
                    <Select Path='Security'>*</Select>
                    <Select Path='System'>*</Select>
                </Query>
            </QueryList>
        </QueryXML>
    </Input>
    
    <Output EventOut>
        Module      om_http
        URL         http://endpoint.io:443
        ContentType application/json
        Exec        set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); to_json();
    </Output>
    
    <Route 1>
        Path    EventIn => EventOut
    </Route>

    This is the output that ElsaticSearch Service is receiving for the EventReceivedTime and EventTime:

    • EventReceivedTime string
    • EventTime string

  • sa's picture

    Note: before proceeding, check if you have the right mapping in place (i.e. EventTime is mapped as a date)

     curl -XGET 'localhost:9200/_template'

    The output should contain something like this:

    mappings":{"_default_":{"properties":{"EventTime":{"format":"YYYY-MM-dd HH:mm:ss","type":"date"}}

     

    If your mapping looks OK, then read on...

    I tried to run a quick test with your config, but unfortunately my Elasticsearch is running in developer mode, so I cannot address it directly from an outside source (long story short: it's running in a non-privileged container) and thus I cannot access it directly from a Windows 2012 box.

    But I can and do use an Nxlog instance in the same container to accept events from the Windows server. What I have is basically a KEN stack (Kibana, Elasticsearch, Nxlog) in a container receiving om_tcp (binary) data and feeding it to ES. This way the EventTime field is treated properly as a date data type.

    Nxlog conf on the KEN server:

    <Input in> 
            Module im_tcp
            Host 0.0.0.0
            Port 1514
            InputType Binary
    </Input>

    <Output es>
            Module      om_http
           
    URL         http://localhost:9200
           
    ContentType application/json
            
    Exec        set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); to_json();
    </Output>

    <Route r>
            Path in => es
    </Route>

    I'm not suggesting that this is the right (let alone only) way to go, but you can use a similar config to check if Windows 2016 events are received correctly and if you are in a time crunch wrt your demo, it can be a workaround for your problem.