Pattern DB appears to drop fields
I'm trying to get a PatternDB working correctly, and it looks like I'm getting some fields but not all of them. There's only one pattern that's actually generating extra fields, and even it is dropping the first field (ParsedDate). Not sure what's going on here...
Config file (via file inclusion):
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input vg_tsw_client>
Module im_file
File "C:\Program Files (x86)\Steam\steamapps\common\The Secret World\ClientLog.txt"
Exec if not ($raw_event =~ /Scaleform\.TSWACT/) drop();
Exec parse_syslog();
</Input>
<Input vg_tsw_combat>
Module im_file
File "C:\Program Files (x86)\Steam\steamapps\common\The Secret World\CombatLog-*.txt"
Exec if ($raw_event =~ /Sprinting [VI]+/) drop();
Exec parse_syslog();
</Input>
<Processor vg_tsw_pattern>
Module pm_pattern
PatternFile %ROOT%\conf\SecretWorld\patterndb.xml
</Processor>
<Output vg_tsw_testfile>
Module om_file
File "C:\\ProgramData\\nxlogs\\vg-tsw-logs.log"
Exec to_json();
</Output>
<Route vg_tsw_route>
Path vg_tsw_client, vg_tsw_combat => vg_tsw_pattern => vg_tsw_testfile
</Route>
Pattern DB:
<?xml version='1.0' encoding='UTF-8'?>
<patterndb>
<created>2010-01-01 01:02:03</created>
<version>42</version>
<group>
<name>tswCombat</name>
<id>50284624</id>
<matchfield>
<name>SourceModuleName</name>
<type>exact</type>
<value>vg_tsw_combat</value>
</matchfield>
<pattern>
<id>1000</id>
<name>basic combat swing</name>
<matchfield>
<name>Message</name>
<type>regexp</type>
<!-- [00:00:28] (Critical) Solomon County Cop's Spray and Pray hits (Normal) Ravenous Horde for 522 physical damage. (Normal) -->
<value>^\[([^\]]+)\] ((?:\(Critical\) |\(Normal\) )?)(.+?'s|Your) (.+?) hits \((Normal|Glancing)\) (.*?) for (\d+) (physical|magical) damage. \((Normal|Penetrated|Blocked)\)</value>
<capturedfield>
<name>ParsedTime</name>
<type>datetime</type>
</capturedfield>
<capturedfield>
<name>CriticalHit</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>AttackerName</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>AttackName</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>Glancing</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>VictimName</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>Damage</name>
<type>integer</type>
</capturedfield>
<capturedfield>
<name>DamageType</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>BlockOrPen</name>
<type>string</type>
</capturedfield>
</matchfield>
<set>
<field>
<name>type</name>
<value>Swing</value>
<type>string</type>
</field>
</set>
</pattern>
</group>
<group>
<name>tswClient</name>
<id>50284625</id>
<matchfield>
<name>SourceModuleName</name>
<type>exact</type>
<value>vg_tsw_client</value>
</matchfield>
<pattern>
<id>2000</id>
<name>tswact load plugin</name>
<matchfield>
<name>Message</name>
<type>regexp</type>
<!-- [2017-02-10 05:47:07Z #3886] [ID:0] ERROR: Scaleform.TSWACT - TSWACT Loaded for |Sheriban| -->
<value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - TSWACT Loaded for - \|(\w+)\|</value>
<capturedfield>
<name>ParsedTime</name>
<type>string</type>
</capturedfield>
<capturedfield>
<name>PlayerName</name>
<type>string</type>
</capturedfield>
</matchfield>
<set>
<field>
<name>type</name>
<value>TswactLoaded</value>
<type>string</type>
</field>
</set>
</pattern>
<pattern>
<id>2001</id>
<name>tswact load playfield</name>
<matchfield>
<name>Message</name>
<type>regexp</type>
<!-- [2017-02-10 05:47:07Z #3886] [ID:0] ERROR: Scaleform.TSWACT - Playfield - |Kingsmouth Town| -->
<value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - Playfield - \|(\w+)\|</value>
<capturedfield>
<name>ParsedTime</name>
<type>datetime</type>
</capturedfield>
<capturedfield>
<name>ZoneName</name>
<type>string</type>
</capturedfield>
</matchfield>
<set>
<field>
<name>type</name>
<value>SetZoneName</value>
<type>string</type>
</field>
</set>
</pattern>
<pattern>
<id>2002</id>
<name>tswact enter combat</name>
<matchfield>
<name>Message</name>
<type>regexp</type>
<!-- [2017-02-10 05:00:22Z #10910] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Sprinting VI:Elemental Force:Third Degree :World Domination| -->
<value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - Enter combat - \|(\w+)\|</value>
<capturedfield>
<name>ParsedTime</name>
<type>datetime</type>
</capturedfield>
<capturedfield>
<name>PlayerName</name>
<type>string</type>
</capturedfield>
</matchfield>
<set>
<field>
<name>type</name>
<value>EnterCombat</value>
<type>string</type>
</field>
</set>
<exec>
$TestField = 'testValue';
</exec>
</pattern>
</group>
</patterndb>
Some of the output I'm getting:
{"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] Your Pop Shot hits (Normal) Undead Islander for 1437 physical damage. (Normal)","CriticalHit":"","AttackerName":"Your","AttackName":"Pop Shot","Glancing":"Normal","VictimName":"Undead Islander","Damage":1437,"DamageType":"physical","BlockOrPen":"Normal","PatternID":1000,"PatternName":"basic combat swing","type":"Swing"}
{"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] (Critical) Your Pop Shot hits (Normal) Undead Islander for 2965 physical damage. (Penetrated)","CriticalHit":"(Critical) ","AttackerName":"Your","AttackName":"Pop Shot","Glancing":"Normal","VictimName":"Undead Islander","Damage":2965,"DamageType":"physical","BlockOrPen":"Penetrated","PatternID":1000,"PatternName":"basic combat swing","type":"Swing"}
{"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] You gain buff Live Wire"}
{"EventReceivedTime":"2017-02-10 11:45:01","SourceModuleName":"vg_tsw_client","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:01","Hostname":"shepard","Message":"[2017-02-10 16:45:01Z #18498] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban|"}
{"EventReceivedTime":"2017-02-10 11:45:10","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:10","Hostname":"shepard","Message":"[11:45:10] Buff Live Wire terminated."}
Some of the vg_tsw_combat input file:
[11:45:00] Your One in the Chamber hits (Normal) Undead Islander for 231 physical damage. (Normal)
[11:45:00] Buff Sudden Return terminated on Undead Islander.
[11:45:00] Buff One in the Chamber terminated on Undead Islander.
[11:45:00] You gained 146 XP.
[11:45:00] Undead Islander died.
[11:45:00] Your Sudden Return hits (Normal) Undead Islander for 259 physical damage. (Normal)
[11:45:00] Your Pop Shot hits (Normal) Undead Islander for 2045 physical damage. (Penetrated)
[11:45:00] Your Pop Shot hits (Normal) Undead Islander for 2175 physical damage. (Penetrated)
[11:45:00] Your Pop Shot hits (Normal) Undead Islander for 1437 physical damage. (Normal)
[11:45:00] (Critical) Your Pop Shot hits (Normal) Undead Islander for 2965 physical damage. (Penetrated)
[11:45:00] You gain buff Live Wire
[11:45:02] You start using Sprinting VI.
[11:45:03] You gain buff Sprinting VI
[11:45:03] You successfully used Sprinting VI.
[11:45:10] Buff Live Wire terminated.
Some of the vg_tsw_client input:
[2017-02-10 16:33:43Z #6790] [ID:0] ERROR: Scaleform.TSWACT - TSWACT Loaded for |Sheriban|
[2017-02-10 16:33:43Z #6790] [ID:0] ERROR: Scaleform.TSWACT - Playfield - |The Savage Coast|
[2017-02-10 16:34:12Z #7313] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:World Domination|
[2017-02-10 16:34:14Z #7373] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban|
[2017-02-10 16:39:06Z #10609] [ID:0] ERROR: MagicCommand - Trying to prepone the execute timeline to the pass. Spell:7760057
[2017-02-10 16:39:06Z #10624] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Elemental Force:World Domination|
[2017-02-10 16:39:08Z #10655] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban|
[2017-02-10 16:44:58Z #18330] [ID:0] ERROR: MagicCommand - Trying to prepone the execute timeline to the pass. Spell:7760057
[2017-02-10 16:44:59Z #18388] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Elemental Force:World Domination|
[2017-02-10 16:45:01Z #18498] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban|
Any ideas?
OK, I tried formatting this with indents and horizontal lines, but neither of those stuck. Sorry.
