1
answer

I'm attempting to demo xnlog and running into a problem where the Windows Server 2016 event logs are being sent to AWS ElasticSearch Service with the EventTime being a string. This basically renders it impossible to index the logs, as the Kibana board requires a time-field name and is not recongizing the string as a datetime.  Any suggestions on this, or is this a potential bug with Server 2016?

AskedJanuary 6, 2017 - 7:08pm

Answer (1)

It's very unlikely that this is due to Windows 2016. Perhaps there is something in your config that converts it to a string but it's hard to tell the cause without that.

AnsweredJanuary 8, 2017 - 5:26pm

Comments (3)

  • chris.bowen's picture

    Here's my current configuration:

    <Extension syslog>
    Module xm_syslog
    </Extension>

    <Extension json>
    Module xm_json
    </Extension>

    <Input EventIn>
    Module im_msvistalog
    ReadFromLast TRUE
    <QueryXML>
    <QueryList>
    <Query Id='1'>
    <Select Path='Application'>*</Select>
    <Select Path='Security'>*</Select>
    <Select Path='System'>*</Select>
    </Query>
    </QueryList>
    </QueryXML>
    </Input>

    <Output EventOut>
    Module om_http
    URL http://endpoint.io:443
    ContentType application/json
    Exec set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); to_json();
    </Output>

    <Route 1>
    Path EventIn => EventOut
    </Route>

    This is the output that ElsaticSearch Service is receiving for the EventReceivedTime and EventTime:

    • EventReceivedTime string
    • EventTime string

    January 9, 2017 - 10:45pm
  • sa's picture

    Note: before proceeding, check if you have the right mapping in place (i.e. EventTime is mapped as a date)

     curl -XGET 'localhost:9200/_template'

    The output should contain something like this:

    mappings":{"_default_":{"properties":{"EventTime":{"format":"YYYY-MM-dd HH:mm:ss","type":"date"}}

     

    If your mapping looks OK, then read on...

    I tried to run a quick test with your config, but unfortunately my Elasticsearch is running in developer mode, so I cannot address it directly from an outside source (long story short: it's running in a non-privileged container) and thus I cannot access it directly from a Windows 2012 box.

    But I can and do use an Nxlog instance in the same container to accept events from the Windows server. What I have is basically a KEN stack (Kibana, Elasticsearch, Nxlog) in a container receiving om_tcp (binary) data and feeding it to ES. This way the EventTime field is treated properly as a date data type.

    Nxlog conf on the KEN server:

    <Input in> 
            Module im_tcp
            Host 0.0.0.0
            Port 1514
            InputType Binary
    </Input>

    <Output es>
            Module      om_http
            URL         http://localhost:9200
            ContentType application/json
            Exec        set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); to_json();
    </Output>

    <Route r>
            Path in => es
    </Route>

    I'm not suggesting that this is the right (let alone only) way to go, but you can use a similar config to check if Windows 2016 events are received correctly and if you are in a time crunch wrt your demo, it can be a workaround for your problem.

     

    January 10, 2017 - 1:44pm
  • chris.bowen's picture

    Thank you.  I was missing the default mappings in the ElasticSearch.  Once I added that, everything seems to be working as expected now.

    January 10, 2017 - 8:24pm