6
responses

Hello,

As the name entices, I am getting multiple errors with XML files.

Here is my conf:
<Extension xml>
Module xm_xml
</Extension>

<Extension json>
Module xm_json
</Extension>

<Input in>
Module im_file
File "C:\\Users\\administrator\\Desktop\\2016.xml"
SavePos FALSE
ReadFromLast FALSE
Exec parse_xml();
Exec to_json();
</Input>

<Output out>
Module om_file
File "C:\\Users\\administrator\\Desktop\\testxml.txt"
</Output>

Here are the errors:
2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20
in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
orted; XML parse error at line 1: no element found
2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20
in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
orted; XML parse error at line 1: not well-formed (invalid token)

Thanks for your time.

AskedSeptember 10, 2018 - 10:16pm

Comments (1)

  • motts's picture

    Here is a copy of the xml file:

    <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
    <Obj RefId="0">
    <TN RefId="0">
    <T>System.Diagnostics.EventLogEntry#Security/Microsoft-Windows-Security-Auditing/4673</T>
    <T>System.Diagnostics.EventLogEntry#Security/Microsoft-Windows-Security-Auditing</T>
    <T>System.Diagnostics.EventLogEntry</T>
    <T>System.ComponentModel.Component</T>
    <T>System.MarshalByRefObject</T>
    <T>System.Object</T>
    </TN>
    <ToString>System.Diagnostics.EventLogEntry</ToString>
    <Props>
    <S N="MachineName">testserver</S>
    <BA N="Data" />
    <I32 N="Index">447206</I32>
    <S N="Category">(13056)</S>
    <I16 N="CategoryNumber">13056</I16>
    <I32 N="EventID">4673</I32>
    <Obj N="EntryType" RefId="1">
    <TN RefId="1">
    <T>System.Diagnostics.EventLogEntryType</T>
    <T>System.Enum</T>
    <T>System.ValueType</T>
    <T>System.Object</T>
    </TN>
    <ToString>SuccessAudit</ToString>
    <I32>8</I32>
    </Obj>
    <S N="Message">A privileged service was called._x000D__x000A__x000D__x000A_Subject:_x000D__x000A__x0009_Security ID:_x0009__x0009_S-1-5-21-26028188-150678075-188441444-157239_x000D__x000A__x0009_Account Name:_x0009__x0009_testaccount_x000D__x000A__x0009_Account Domain:_x0009__x0009_testdomain_x000D__x000A__x0009_Logon ID:_x0009__x0009_0x2053a6e4_x000D__x000A__x000D__x000A_Service:_x000D__x000A__x0009_Server:_x0009_Security_x000D__x000A__x0009_Service Name:_x0009_-_x000D__x000A__x000D__x000A_Process:_x000D__x000A__x0009_Process ID:_x0009_0x1770_x000D__x000A__x0009_Process Name:_x0009_C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe_x000D__x000A__x000D__x000A_Service Request Information:_x000D__x000A__x0009_Privileges:_x0009__x0009_SeCreateGlobalPrivilege</S>
    <S N="Source">Microsoft-Windows-Security-Auditing</S>
    <Obj N="ReplacementStrings" RefId="2">
    <TN RefId="2">
    <T>System.String[]</T>
    <T>System.Array</T>
    <T>System.Object</T>
    </TN>
    <LST>
    <S>S-1-5-21-26028188-150678075-188441444-157239</S>
    <S>testaccount</S>
    <S>testdomain</S>
    <S>0x2053a6e4</S>
    <S>Security</S>
    <S>-</S>
    <S>SeCreateGlobalPrivilege</S>
    <S>0x1770</S>
    <S>C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</S>
    </LST>
    </Obj>
    <I64 N="InstanceId">4673</I64>
    <DT N="TimeGenerated">2018-08-14T08:34:37-04:00</DT>
    <DT N="TimeWritten">2018-08-14T08:34:37-04:00</DT>
    <Nil N="UserName" />
    <Nil N="Site" />
    <Nil N="Container" />
    </Props>
    <MS>
    <I32 N="EventID">4673</I32>
    </MS>
    </Obj>
    </Objs>

Answer (1)

Your input is a multiline XML. You will need to also use xm_multiline. Something like the following.

<Extension multiline_parser>
    Module          xm_multiline
    HeaderLine      /^<Objs/
    EndLine         /^</Objs>/
</Extension>

<Input in>
    Module im_file
    File "file.xml"
    SavePos FALSE
    ReadFromLast FALSE
    InputType multiline_parser
    Exec parse_xml();
    Exec to_json();
</Input>

Should result in the following ( with JSON directive PrettyPrint True for readability)

{
    "EventReceivedTime": "2018-09-10 17:33:24.986183-05:00",
    "SourceModuleName": "in",
    "SourceModuleType": "im_file",
    "Objs.Obj.TN.T": "System.Object",
    "Objs.Obj.ToString": "System.Diagnostics.EventLogEntry",
    "Objs.Obj.Props.S": "Microsoft-Windows-Security-Auditing",
    "Objs.Obj.Props.I32": "4673",
    "Objs.Obj.Props.I16": "13056",
    "Objs.Obj.Props.Obj.TN.T": "System.Object",
    "Objs.Obj.Props.Obj.ToString": "SuccessAudit",
    "Objs.Obj.Props.Obj.I32": "8",
    "Objs.Obj.Props.Obj.LST.S": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
    "Objs.Obj.Props.I64": "4673",
    "Objs.Obj.Props.DT": "2018-08-14T08:34:37-04:00",
    "Objs.Obj.MS.I32": "4673"
}

Comments (4)

  • motts's picture

    Here it is, just like how you have it, but I am still getting the same errors

    <Extension multi>
    Module xm_multiline
    Headerline /^<Objs/
    Endline /^</Objs>/
    </Extension>

    <Input in>
    Module im_file
    File "C:\\Users\\administrator\\Desktop\\2016.xml"
    SavePos FALSE
    ReadFromLast FALSE
    Exec parse_xml();
    Exec to_json();
    </Input>

    Errors
    2018-09-11 10:10:21 ERROR procedure 'parse_xml' failed at line 40, character 20
    in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
    orted; XML parse error at line 1: no element found
    2018-09-11 10:10:21 ERROR procedure 'parse_xml' failed at line 40, character 20
    in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
    orted; XML parse error at line 1: not well-formed (invalid token)

  • motts's picture

    Also, just checked the output file and it's partly in another language. It appears the fields are fine, but the values are messed up. Probably a charset things, but i wouldn't think that would cause the parse_xml errors.

    <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">਍ <Obj RefId="0">਍ <TN RefId="0">਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </TN>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ <Props>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ <Obj N="EntryType" RefId="1">਍ <TN RefId="1">਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </TN>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </Obj>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ <Obj N="ReplacementStrings" RefId="2">਍ <TN RefId="2">਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </TN>਍ <LST>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </LST>਍ </Obj>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </Props>਍ <MS>਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴹㄱㄠ㨰㤰㌺∲∬潓牵散潍畤敬慎敭㨢椢≮∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ </MS>਍ </Obj>਍

  • Zhengshi's picture
    (NXLog)

    Charset could cause the element errors. If it can not read the opening and closing tags, it will not parse them proper.

    I would start with adding xm_charconv and convert_fields("AUTO", "utf-8");
    https://nxlog.co/documentation/nxlog-user-guide#xm_charconv

  • motts's picture

    I tried this per the example of the guide:

    <Extension xml>
    Module xm_xml
    </Extension>

    <Extension json>
    Module xm_json
    </Extension>

    <Extension charcon>
    Module xm_charconv
    #AutodetectCharsets utf-8, ucs-2le
    </Extension>

    <Extension multi>
    Module xm_multiline
    Headerline /^<Objs/
    Endline /^</Objs>/
    </Extension>

    <Input in>
    Module im_file
    File "C:\\Users\\administrator\\Desktop\\2016.xml"
    Exec convert_fields("auto", "utf-8");
    SavePos FALSE
    ReadFromLast FALSE
    Exec parse_xml();
    Exec to_json();
    </Input>

    but I still get those same 2 errors multiple times

    2018-09-11 12:41:06 ERROR procedure 'parse_xml' failed at line 46, character 20
    in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
    orted; XML parse error at line 1: no element found
    2018-09-11 12:41:06 ERROR procedure 'parse_xml' failed at line 46, character 20
    in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
    orted; XML parse error at line 1: not well-formed (invalid token)

    I comment out the AutodetectCharsets since you didn't mention that and I am reduced just to one error:

    2018-09-11 12:43:32 ERROR procedure 'parse_xml' failed at line 46, character 20
    in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
    orted; XML parse error at line 1: no element found

    However, my output is just these squares.

    ਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍਍

    The XML I am using is in "UCS-2 LE BOM" and from what I could find, using ucs-2le should have worked in the autodetectcharset. Maybe it's something else?