Hello. I have a question.

I get multiline messages

how can I combine into a single line, multiline message ??

for example this message, In this message 4 lines

Jul 21 17:59:10 <14> 1 2016-07-04T00: 53: 02.000000 + 03: 00 node = sec-sflow type = SYSCALL msg = audit (1467579182.055: 3248181): arch = 111

2 syscall = success = yes exit = 4 a0 = 7fc7783127a8 a1 = 2 a2 = a3 = 0 8 items = 1 ppid = 11013 pid = 30363 auid = 0 0 uid = gid = 0 = 0 euid

suid = 0 fsuid = 0 = 0 egid sgid = 0 = 0 fsgid tty = (none) ses = 28 comm = "sshd"

exe = "/ usr / sbin / sshd" key = "root_action"

Thank!

I am trying to use the multlog module in order to start ingesting a custom log:

I have the following regex: \^(\d{2}|\d).(\d{2}|\d).(\d{4})\s(\d\d|\d):(\d\d|\d):(\d\d|\d)\s(AM|PM).\[(.*)\](.*) 

This works in a regex test; however I cannot get it to work with the log file that looks something like this

9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-17] GetStatus for IP: 192.168.0.231 on port: 5016

9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-47] <TRANSACTION>
  <FUNCTION_TYPE>SECONDARYPORT</FUNCTION_TYPE>
  <COMMAND>STATUS</COMMAND>
  <MAC_LABEL>P_061</MAC_LABEL>
  <MAC>az4FMuLbvrPz720bBeKWz3c+zBh6MsKVo4nJEW96B04=</MAC>
  <COUNTER>217</COUNTER>
</TRANSACTION>

9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-57] <RESPONSE>
  <RESPONSE_TEXT>Operation SUCCESSFUL</RESPONSE_TEXT>
  <RESULT>OK</RESULT>
  <RESULT_CODE>-1</RESULT_CODE>
  <TERMINATION_STATUS>SUCCESS</TERMINATION_STATUS>
  <COUNTER>217</COUNTER>
  <SECONDARY_DATA>10</SECONDARY_DATA>
  <SERIAL_NUMBER>285498613</SERIAL_NUMBER>
</RESPONSE>

9/10/2015 11:29:16 AM [0-1-1-LandingPage.xaml.cs-49] POS opened

However when running the nxlog.conf for this I am getting the following error 

2015-09-15 08:00:43 ERROR couldn't parse expression at line 12, character 13 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; invalid character: '\'

I am unsure what i need to do in order to get this correct; does anyone have any insight or resources I should further explore. Is there a REGEX specific doc for NXLOG? 

Hi,

I am trying to parse a log4net file into json.

Here's my sample log4net:

----------------

2015-01-27 01:06:18,859 [7] ERROR Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider [(null)] - Get taxonomy Type Failed for Tools
2015-01-27 06:34:31,051 [26] ERROR www.Status404 [(null)] - ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20
    UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36
    HostAddress: 192.168.10.2
    RequestUrl: /ErrorPages/404.aspx
    MachineName: QA01
    Raw Url:/undefined/
    Referrer: http://qa1.www.something.com/toolset.aspx

2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled

Now I am using xm_multiline to capture each log entries.

----------------

<Extension multiline>
    Module        xm_multiline
    HeaderLine    /^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/
    EndLine        /\r?\n\r?\n^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/
</Extension>

I use a regex to capture the timestamp as the header then I use a regex to capture twice newline then the next timestamp as endline. However it still treat the second and last entry as ONE log entry.

Here's the output:

----------------

{  
   "EventReceivedTime":"2015-01-27 01:06:35",
   "SourceModuleName":"log4net",
   "SourceModuleType":"im_file",
   "time":"2015-01-27 01:06:18,859",
   "thread":"7",
   "level":"ERROR",
   "logger":"Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider",
   "ndc":"(null)",
   "message":"Get taxonomy Type Failed for Tools"
}{  
   "EventReceivedTime":"2015-01-27 06:34:35",
   "SourceModuleName":"log4net",
   "SourceModuleType":"im_file",
   "time":"2015-01-27 06:34:31,051",
   "thread":"26",
   "level":"ERROR",
   "logger":"www.Status404",
   "ndc":"(null)",
   "message":"  ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20\r\n  UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 
Safari/537.36\r\n  HostAddress: 192.168.10.2\r\n  RequestUrl: /ErrorPages/404.aspx\r\n  MachineName: QA01\r\n  
Raw Url:/undefined/\r\n  Referrer: http://qa1.www.something.com/toolset.aspx\r\n\r\n2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled"
}

I used this to produce that output:

----------------

Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- (.*)/s \
                { \
                    $time = $1; \
                    $thread = $2; \
                    $level = $3; \
                    $logger = $4; \
                    $ndc = $5; \
                    $message = $6; \
                    to_json(); \
                } \
                else \
                { \
                    drop(); \
                }

I've also tried to tweak it by using this to avoid the combining the last two entries as one. However I am not able to get the last entry anymore.

----------------

Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- ([\s\S]*?)(\r?\n\r?\n|$)/ \
                { \
                    $time = $1; \
                    $thread = $2; \
                    $level = $3; \
                    $logger = $4; \
                    $ndc = $5; \
                    $message = $6; \
                    to_json(); \
                } \
                else \
                { \
                    drop(); \
                }​