I'm trying to configure an Exclude in im_file. I want to exclude logs with a date and number at the end of the filename. These files are already processed.Config:<Input PPS>  Module im_file  InputType multi_PPS  File '%LOGDIR%\\*.log'  # PPSPortaalManager_24-05-24_2.log  Exclude '*_??-??-??_*.log'  #. I've tried several notations, eg. double quotes instead of single, escaping the questionmarks. But none of them work. Searching for examples leads to 1 of 2 examples, which are very common examples, and none of them with wildcards.Using nxlog-ce-3.2.2329 .Has anyone a good, and working, example how to user Exclude in im_file?

I've been testing sending logs directly to Sentinel and am having a problem with NXLOG not liking the CA. The error is not one I'm finding a lot of online help with.“no certificate or crl found” appears in the the log file, repeatedly.Maybe an issue with the CA I'm pointing to? Is there a specific one for Azure Sentinel that I'm overlooking, and if so, where is that obtained?Thanks in advance.--B

 Hello there i try to forward logs radius to my elastic siem , i got this error when executing nxlog.exe -f  :2024-10-07 11:16:37 INFO nxlog-ce-3.2.2329 started2024-10-07 11:16:37 INFO reconnecting in 1 seconds2024-10-07 11:16:37 ERROR apr_sockaddr_info failed for 172.19.14.51  # IP du serveur distant:514; Unknow Host. Here is the config (without ip for privacy) : <Extension _syslog>   Module      xm_syslog</Extension><Extension xml>   Module      xm_xml</Extension><Input radius_log>        Module      im_file      File        "D:\NPS\NPS Logs\IN2410.log"     SavePos     TRUE       ReadFromLast TRUE       PollInterval 1       Exec        parse_xml("/Event"); </Input><Output remote_syslog>        Module      om_udp   Host         # IP SRV FORWARD        Port        514</Output><Route radius_to_remote>     Path        radius_log => remote_syslog</Route> Is it possible to get everything in the source folder not just one ? Because the name change everymonth , example : october  :IN2410.logNovember: IN2411.log December: IN2412.log   etc Please  Thnaks a lot

Hello All, In NXlog Qradar windows configuration , we have setup forwarded events as well but the for the forwarded events the source is coming as host instead of client meachine or actual source. I would request your help in fixing this. I am using the below configuration define ROOT     C:\Program Files\nxlogdefine CERTDIR  %ROOT%\certdefine CONFDIR  %ROOT%\conf\nxlog.ddefine LOGDIR   %ROOT%\datainclude %CONFDIR%\\*.confdefine LOGFILE  %LOGDIR%\nxlog.logLogFile %LOGFILE%Moduledir %ROOT%\modulesCacheDir  %ROOT%\dataPidfile   %ROOT%\data\nxlog.pidSpoolDir  %ROOT%\data<Extension _syslog>   Module      xm_syslog</Extension><Extension _charconv>   Module      xm_charconv   AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32</Extension><Extension _exec>   Module      xm_exec</Extension><Extension _fileop>   Module      xm_fileop   # Check the size of our log file hourly, rotate if larger than 5MB   <Schedule>       Every   1 hour       Exec    if (file_exists('%LOGFILE%') and \                  (file_size('%LOGFILE%') >= 5M)) \                   file_cycle('%LOGFILE%', 8);   </Schedule>   # Rotate our log file every week on Sunday at midnight   <Schedule>       When    @weekly       Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);   </Schedule></Extension># Snare compatible example configuration# Collecting event log<Input in>    Module      im_msvistalog<QueryXML>       <QueryList>           <Query Id='0'>               <Select Path='Application'>*</Select>               <Select Path='Security'>*[System/Level<4]</Select>               <Select Path='System'>*</Select>   <Select Path='ForwardedEvents'>*</Select>                                </Query>       </QueryList>   </QueryXML>   <Exec>       if $Category == undef $Category = 0;       if $EventType == 'CRITICAL'       {           $EventTypeNum = 1;           $EventTypeStr = "Critical";       }       else if $EventType == 'ERROR'       {           $EventTypeNum = 2;           $EventTypeStr = "Error";       }       else if $EventType == 'INFO'       {           $EventTypeNum = 4;           $EventTypeStr = "Informational";       }       else if $EventType == 'WARNING'       {           $EventTypeNum = 3;           $EventTypeStr = "Warning";       }       else if $EventType == 'VERBOSE'       {           $EventTypeNum = 5;           $EventTypeStr = "Verbose";       }       else       {           $EventTypeNum = 0;           $EventTypeStr = "Audit";       }       if $OpcodeValue == 0 $Opcode = "Info";       if $TaskValue == 0 $TaskValue = "None";       $EpochTime = string(integer($EventTime));       $EpochTime =~ /^(?<sec>\d+)(?<ms>\d{6})$/;       $EpochTime = $sec;       if $TaskValue == 12288 { $TaskStr = "SE_ADT_SYSTEM_SECURITYSTATECHANGE"; }       else if $TaskValue == 12289 { $TaskStr = "SE_ADT_SYSTEM_SECURITYSUBSYSTEMEXTENSION"; }       else if $TaskValue == 12290 { $TaskStr = "SE_ADT_SYSTEM_INTEGRITY"; }       else if $TaskValue == 12291 { $TaskStr = "SE_ADT_SYSTEM_IPSECDRIVEREVENTS"; }       else if $TaskValue == 12292 { $TaskStr = "SE_ADT_SYSTEM_OTHERS"; }       else if $TaskValue == 12544 { $TaskStr = "SE_ADT_LOGON_LOGON"; }       else if $TaskValue == 12545 { $TaskStr = "SE_ADT_LOGON_LOGOFF"; }       else if $TaskValue == 12546 { $TaskStr = "SE_ADT_LOGON_ACCOUNTLOCKOUT"; }       else if $TaskValue == 12547 { $TaskStr = "SE_ADT_LOGON_IPSECMAINMODE"; }       else if $TaskValue == 12548 { $TaskStr = "SE_ADT_LOGON_SPECIALLOGON"; }       else if $TaskValue == 12549 { $TaskStr = "SE_ADT_LOGON_IPSECQUICKMODE"; }       else if $TaskValue == 12550 { $TaskStr = "SE_ADT_LOGON_IPSECUSERMODE"; }       else if $TaskValue == 12551 { $TaskStr = "SE_ADT_LOGON_OTHERS"; }       else if $TaskValue == 12552 { $TaskStr = "SE_ADT_LOGON_NPS"; }       else if $TaskValue == 12553 { $TaskStr = "SE_ADT_LOGON_CLAIMS"; }       else if $TaskValue == 12554 { $TaskStr = "SE_ADT_LOGON_GROUPS"; }       else if $TaskValue == 12800 { $TaskStr = "SE_ADT_OBJECTACCESS_FILESYSTEM"; }       else if $TaskValue == 12801 { $TaskStr = "SE_ADT_OBJECTACCESS_REGISTRY"; }       else if $TaskValue == 12802 { $TaskStr = "SE_ADT_OBJECTACCESS_KERNEL"; }       else if $TaskValue == 12803 { $TaskStr = "SE_ADT_OBJECTACCESS_SAM"; }       else if $TaskValue == 12804 { $TaskStr = "SE_ADT_OBJECTACCESS_OTHER"; }       else if $TaskValue == 12805 { $TaskStr = "SE_ADT_OBJECTACCESS_CERTIFICATIONAUTHORITY"; }       else if $TaskValue == 12806 { $TaskStr = "SE_ADT_OBJECTACCESS_APPLICATIONGENERATED"; }       else if $TaskValue == 12807 { $TaskStr = "SE_ADT_OBJECTACCESS_HANDLE"; }       else if $TaskValue == 12808 { $TaskStr = "SE_ADT_OBJECTACCESS_SHARE"; }       else if $TaskValue == 12809 { $TaskStr = "SE_ADT_OBJECTACCESS_FIREWALLPACKETDROPS"; }       else if $TaskValue == 12810 { $TaskStr = "SE_ADT_OBJECTACCESS_FIREWALLCONNECTION"; }       else if $TaskValue == 12811 { $TaskStr = "SE_ADT_OBJECTACCESS_DETAILEDFILESHARE"; }       else if $TaskValue == 12812 { $TaskStr = "SE_ADT_OBJECTACCESS_REMOVABLESTORAGE"; }       else if $TaskValue == 12813 { $TaskStr = "SE_ADT_OBJECTACCESS_CBACSTAGING"; }       else if $TaskValue == 13056 { $TaskStr = "SE_ADT_PRIVILEGEUSE_SENSITIVE"; }       else if $TaskValue == 13057 { $TaskStr = "SE_ADT_PRIVILEGEUSE_NONSENSITIVE"; }       else if $TaskValue == 13058 { $TaskStr = "SE_ADT_PRIVILEGEUSE_OTHERS"; }       else if $TaskValue == 13312 { $TaskStr = "SE_ADT_DETAILEDTRACKING_PROCESSCREATION"; }       else if $TaskValue == 13313 { $TaskStr = "SE_ADT_DETAILEDTRACKING_PROCESSTERMINATION"; }       else if $TaskValue == 13314 { $TaskStr = "SE_ADT_DETAILEDTRACKING_DPAPIACTIVITY"; }       else if $TaskValue == 13315 { $TaskStr = "SE_ADT_DETAILEDTRACKING_RPCCALL"; }       else if $TaskValue == 13316 { $TaskStr = "SE_ADT_DETAILEDTRACKING_PNPACTIVITY"; }       else if $TaskValue == 13317 { $TaskStr = "SE_ADT_DETAILEDTRACKING_TOKENRIGHTADJ"; }       else if $TaskValue == 13568 { $TaskStr = "SE_ADT_POLICYCHANGE_AUDITPOLICY"; }       else if $TaskValue == 13569 { $TaskStr = "SE_ADT_POLICYCHANGE_AUTHENTICATIONPOLICY"; }       else if $TaskValue == 13570 { $TaskStr = "SE_ADT_POLICYCHANGE_AUTHORIZATIONPOLICY"; }       else if $TaskValue == 13571 { $TaskStr = "SE_ADT_POLICYCHANGE_MPSSCVRULEPOLICY"; }       else if $TaskValue == 13572 { $TaskStr = "SE_ADT_POLICYCHANGE_WFPIPSECPOLICY"; }       else if $TaskValue == 13573 { $TaskStr = "SE_ADT_POLICYCHANGE_OTHERS"; }       else if $TaskValue == 13824 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_USERACCOUNT"; }       else if $TaskValue == 13825 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_COMPUTERACCOUNT"; }       else if $TaskValue == 13826 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_SECURITYGROUP"; }       else if $TaskValue == 13827 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_DISTRIBUTIONGROUP"; }       else if $TaskValue == 13828 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_APPLICATIONGROUP"; }       else if $TaskValue == 13829 { $TaskStr = "SE_ADT_ACCOUNTMANAGEMENT_OTHERS"; }       else if $TaskValue == 14080 { $TaskStr = "SE_ADT_DSACCESS_DSACCESS"; }       else if $TaskValue == 14081 { $TaskStr = "SE_ADT_DSACCESS_DSCHANGES"; }       else if $TaskValue == 14082 { $TaskStr = "SE_ADT_DS_REPLICATION"; }       else if $TaskValue == 14083 { $TaskStr = "SE_ADT_DS_DETAILED_REPLICATION"; }       else if $TaskValue == 14336 { $TaskStr = "SE_ADT_ACCOUNTLOGON_CREDENTIALVALIDATION"; }       else if $TaskValue == 14337 { $TaskStr = "SE_ADT_ACCOUNTLOGON_KERBEROS"; }       else if $TaskValue == 14338 { $TaskStr = "SE_ADT_ACCOUNTLOGON_OTHERS"; }       else if $TaskValue == 14339 { $TaskStr = "SE_ADT_ACCOUNTLOGON_KERBCREDENTIALVALIDATION"; }       else if $TaskValue == 65280 { $TaskStr = "SE_ADT_UNKNOWN_SUBCATEGORY"; }       else { $TaskStr = "Unknown[" + $taskValue + "]"; }   if $KeywordsStr == undef {       if $TaskValue == 0 {           $KeywordsStr = 'None';       } else {           $KeywordsStr = '0';       }   }   if $TaskStr == undef {       $TaskStr = $TaskValue;   }   if $EventType == 'AUDIT_SUCCESS' {       $KeywordsStr = "Audit Success";       $EventTypeNum = 8;   } else {       $KeywordsStr = "Audit Failure";       $EventTypeNum = 16;   }       $Message = "AgentDevice=WindowsLog" +           "\tAgentLogFile=" + $Channel +           "\tSource=" + $SourceName +           "\tComputer=" + hostname_fqdn() +           "\tOriginatingComputer=" + host_ip() +           "\tUser=" + $AccountName +           "\tDomain=" + $Domain +           "\tEventIDCode=" + $EventID +           "\tEventType=" + $EventTypeNum +           "\tEventCategory=" + $TaskValue +           "\tRecordNumber=" + $RecordNumber +           "\tTimeGenerated=" + $EpochTime +           "\tTimeWritten=" + $EpochTime +           "\tLevel=" + $EventTypeStr +           "\tKeywords=" + $KeywordsStr +           "\tTask=" + $TaskStr +           "\tOpcode=" + $Opcode +           "\tMessage=" + $Message;       $Hostname = host_ip();       delete($SourceName);       delete($Severity);       delete($SeverityValue);       to_syslog_bsd();   </Exec></Input>

Hi Team,Need help!We're unable to ingest the logs from the file ‘output.json,’ which is 150MB in size. But i am able to send the logs from file ‘output.json’ which is 10MB in size to the forwarder.Below is the NXLog Community Edition Config file. Please help me is there anything to modify the Config file.define ROOT C:\Program Files\nxlogdefine ADCONTEXT_OUTPUT_DESTINATION_ADDRESS <Forwarder IP>define ADCONTEXT_OUTPUT_DESTINATION_PORT <Port>Moduledir   %ROOT%\modulesCacheDir    %ROOT%\dataPidfile         %ROOT%\data\nxlog.pidSpoolDir     %ROOT%\dataLogFile       %ROOT%\data\nxlog.log<Extension json>    Module  xm_json</Extension><Input in_adcontext>   Module    im_file   File         "C:\AD\output.json"   DirCheckInterval 3600   PollInterval 3600</Input><Output out_chronicle_adcontext>   Module    om_tcp   Host       %ADCONTEXT_OUTPUT_DESTINATION_ADDRESS%   Port        %ADCONTEXT_OUTPUT_DESTINATION_PORT%</Output><Route ad_context_to_chronicle>    Path in_adcontext => out_chronicle_adcontext</Route>

Hi,We are using NXLog-CE version 2.10.2150 and we prepare to deploy NX log agent in Windows server 2022.Is Nxlog 2.10.2150 supporting log collection for Windows server 2022?If the version is not supported, what is the minimum NX log agent to support Windows server 2022?Thanks.

Hi NXLOG Community,the NXLOG Process 100% CPU Load after Server restart on Windows Systems.The Problemcause is simple but i dont understand why.A Handfull of Servers whit much logfiles (nxlogconfiguration use Wildcard for logfolder(s)).The Logfiles are written in the Cachefile and looks okay. But it feels like digs through all log files.Small footage of a Configuration:<Input catalina> Module im_file File 'C:\Tomcat\logs\\catalina*.log' Recursive True PollInterval 30 SavePos True CloseWhenIdle True &lt;Exec&gt; #parse date if $raw_event =~ /(\d\d\-\S\S\S-\d\d\d\d \d\d:\d\d:\d\d)/ $EventTime = strptime($1, '%d-%b-%Y%t%H:%M:%S'); # Now set the severity to something custom. This defaults to 'INFO' if unset. if $raw_event =~ /\sSEVERE\s/ $Severity = 'CRITICAL'; else if $raw_event =~ /\sERROR\s/ $Severity = 'ERROR'; else if $raw_event =~ /\sWARN\s/ $Severity = 'WARNING'; else $Severity = 'INFO'; # The facility can be also set, otherwise the default value is 'USER'. $SyslogFacility = 'AUDIT'; &lt;/Exec&gt; </Input>We use the Last CE Version.: nxlog-ce-3.2.2329Thanks, regards Wolfgang 

Have installed nx onprem on ubuntu 22.04.5 LTS and cant login after the install finishes. Not sure how many pods are suppose to run but I see a postgress and vault 1c9df1fc6f5d  nxlogacr.azurecr.io/vault:1.13.3                                3 hours ago  Up 2 hours ago (healthy)              nxlog-1_2_2-vault-1eef5bec91376  nxlogacr.azurecr.io/postgres:16.3-alpine  -c config_file=/e...  3 hours ago  Up 2 hours ago (healthy)              nxlog-1_2_2-postgres-1 Below is what shows listening and I don't see any web services. dp    UNCONN  0       0                                      10.89.0.1:53          0.0.0.0:*      users:(("dnsmasq",pid=1291,fd=4))          udp    UNCONN  0       0                                  127.0.0.53%lo:53          0.0.0.0:*      users:(("systemd-resolve",pid=627,fd=13))  udp    UNCONN  0       0        [fe80::d433:a2ff:fe02:9e0d]%cni-podman1:53             [::]:*      users:(("dnsmasq",pid=1291,fd=10))         tcp    LISTEN  0       32                                     10.89.0.1:53          0.0.0.0:*      users:(("dnsmasq",pid=1291,fd=5))          tcp    LISTEN  0       4096                               127.0.0.53%lo:53          0.0.0.0:*      users:(("systemd-resolve",pid=627,fd=14))  tcp    LISTEN  0       128                                      0.0.0.0:22          0.0.0.0:*      users:(("sshd",pid=750,fd=3))              tcp    LISTEN  0       32       [fe80::d433:a2ff:fe02:9e0d]%cni-podman1:53             [::]:*      users:(("dnsmasq",pid=1291,fd=11))         tcp    LISTEN  0       128                                         [::]:22             [::]:*      users:(("sshd"I see the following error in nxp.logHA Mode                 standbyActive Node Address     <none>Raft Committed Index    31Raft Applied Index      31Error authenticating: error looking up token: Error making API request.URL: GET http://0.0.0.0:8200/v1/auth/token/lookup-selfCode: 500. Errors:I got the following during the install[2024-09-25 00:28:55] [INFO] Vault container ID: 62f8bd5e1e00[2024-09-25 00:28:55] [INFO] Executing command (/init/bootstrap.sh) in container 62f8bd5e1e00...make: *** [Makefile:231: seed-vault] Error 2

Hello,I'm attempting to install the NXLog Platform on-prem on a Ubuntu 24.04 LTS VM, but I am running into the following error on the nxlog-1_2_2-vault-1 container:fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later) WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later) ERROR: unable to select packages: supervisor (no such package): required by: world[supervisor] fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later) WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later) ERROR: unable to select packages: supervisor (no such package): required by: world[supervisor] fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz When accessing this link from my host machine, I am able to download the .tar.gz, so it does not seem to be a network issue. Additionally, from the VM I am able to reach the internet perfectly fine to, e.g., run updates and I can cURL the URL from the VM as well.Any assistance on this would be appreciated!Thanks

Hi, seems there is a bug in im_msvistalog module in nxlog-ce-3.0.2272 for Windows. After service has generated some output, stopping service sometimes give the message box "Broken pipe" and sometimes Nxlog write a malformed configcache.dat and following the start posts an ERROR failed to restore the saved position from bookmark xml (error:15008)" Tested on Windows 2012 R2 and Windows 2019 <Output out> Module om_file File '%LOGDIR%\Output' </Output> <Input _im_eventlog> Module im_msvistalog SavePos TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='Security'>*</Select> </Query> </QueryList> </QueryXML> # Query <QueryList>\ # <Query Id='0'>\ # <Select Path="Security">*</Select>\ # </Query>\ # </QueryList> </Input> #<Output outNull> # Module om_null #</Output> <Route RouteA> Path _im_eventlog => out </Route> This issue make the module not worth to use because not saving the bookmark of the processed registry record, force it to generate same logs from start over each service restart. Best.

HiHere is my configuration. However, only Sysmon events are not working because they contain "/operational: " at the beginning of the message, which causes the events to be parsed incorrectlySo I want to know how to remove "/operational:  "<Extension syslog>   Module          xm_syslog</Extension><Input in>   Module          im_msvistalog   ReadFromLast True    <QueryXML>            <QueryList>  <Query Id="0">   <Select Path="Application">*</Select>   <Select Path="System">*</Select>   <Select Path="Security">*</Select>   <Select Path="Windows PowerShell">*</Select>   <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>  </Query> </QueryList> </QueryXML>   Exec          $UnixTime = integer($EventTime)/1000; </Input><Output devo_relay> Module        om_tcp Host          192.168.29.133Port          13000 I tried Exec if ($SourceName =~ /Microsoft-Windows-Sysmon\/Operational/) { $Message = replace($Message, "/operational: ", ""); }and Exec    if ($Message =~ /\/operational: /) {                     $Message = replace($Message, "/operational: ", "");                 }But all did not work

I'm successfully using this config format with im_file to read logs with todays date in the filename:'\\server.domain\Logs\IN' + strftime(now(), "%y%m%d") + '.log'One of our services writes its log for the previous day at 3am on the next day. The filename has yesterdays date. What's the easiest/neatest/most efficient way of reading this log please?

Hi folks,We have an NXLog CE 3.2.2329 configuration using im_tcp with InputType LineBased, receiving blobs of JSON separated by newlines. The received data is passed to xm_perl and then relayed to an output.Under some conditions where there may be thousands of events per second, there is a very rare chance for one event to be truncated at the end yielding an incomplete json blob.  We've verified that the input data coming from upstream is valid, fully formed JSON - the emitter encoding logs look OK. The preceeding and following lines are also received and parsed successfully. Has anybody else experienced similar behavior? At this point we suspect a bug in NXLog, considering that it occurs very infrequently (2-3 days between recurrences) and it only occurs during periods of relatively high load (normal load is ~150 events per second, load when issue occurs is ~800/s+)Relevant configuration: <Extension gelf>Module      xm_gelf</Extension><Extension perl>Module      xm_perlPerlCode    /etc/nxlog/scripts/parse_mist.pl</Extension><Input mist_tcp>Module      im_tcpHost        0.0.0.0Port        32768InputType   LineBased<Exec>perl_call("parse_mist");if $nxlog_internal_drop_log drop();</Exec></Input><Output graylog>Module      om_udpHost        xPort        12201OutputType  GELF</Output><Route graylog>Path       mist_tcp => graylog</Route>

I've successfully installed the NXLog platform on Ubuntu 22.04.4 LTS, but I'm facing an issue with port 443 not being accessible. I’ve allowed port 443 through the firewall, but I still can't access the web console. I’ve checked the service status, and everything seems fine. Is there any configuration or additional steps I might have missed to get port 443 up and running?Any help would be appreciated. Thanks in advance!@Ayodele@2Emeka Nwankwo 

I have setup with RHEL 7.9 (kernel  3.10.0-1160.108.1.el7.x86_64) / NXLog CE 3.2.2329 / Postgresql 15 vanilla / libdbi 8.4 with configuration:User nxlog Group nxlog include /etc/nxlog/nxlog.d/*.conf LogFile /var/log/nxlog/nxlog.log LogLevel DEBUG <Extension charconv> Module xm_charconv </Extension> <Input sqlite3_dbi> Module im_dbi SQL SELECT * FROM vw_tp_message Driver sqlite3 Option dbname term_refdata.db Option sqlite3_dbdir /app/data/monitoring Option sqlite3_timeout 1000 PollInterval 5 SavePos TRUE </Input> <Output postgres_dbi> Module om_dbi SQL INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ($SyslogFacility, $SyslogSeverity, $Hostname, '$EventTime', $SourceName, $Message) Driver pgsql Option host /var/run/postgresql Option username srv_nxlog Option dbname MSGLog Exec $Message = convert($Message, "windows-1251", "utf-8"); </Output> <Route sqlite_postgres> Path sqlite3_dbi => postgres_dbi </Route>There are only 2 records in sqlite database:$ sqlite3 -header /app/data/monitoring/term_refdata.db 'select * from vw_tp_message' id|SyslogFacility|SyslogSeverity|Hostname|EventTime|SourceName|Message 1293441|USER|INFO|SERVER1-D1-CL|2024-09-12 00:18:22.540|tp_msg|Some text in Windows1251 encoding 1293442|USER|INFO|SERVER1-D1-CL|2024-09-12 00:41:04.677|tp_msg|Another text in Windows1251 encodingAnd this is what I get in nxlog.log:... 2024-09-12 16:54:59 DEBUG logdata missing or undef 'EventTime', setting to NULL 2024-09-12 16:54:59 DEBUG om_dbi SQL: INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Some text in Windows1251 encoding') 2024-09-12 16:54:59 ERROR [om_dbi.c:256/om_dbi_write()] -;[om_dbi.c:85/om_dbi_error()] om_dbi failed to execute SQL statement "INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Some text in Windows1251 encoding')". ERROR: invalid input syntax for type timestamp: "NULL";LINE 1: ... VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 't...; ^;[errorcode: 0] ... 2024-09-12 16:54:59 DEBUG logdata missing or undef 'EventTime', setting to NULL 2024-09-12 16:54:59 DEBUG om_dbi SQL: INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Р?нициализация СЏРґСЂР° после перезапуска') 2024-09-12 16:54:59 ERROR [om_dbi.c:256/om_dbi_write()] -;[om_dbi.c:85/om_dbi_error()] om_dbi failed to execute SQL statement "INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Р?нициализация СЏРґСЂР° после перезапуска')". ERROR: invalid input syntax for type timestamp: "NULL";LINE 1: ... VALUES ('USER', 'INFO', 'ABACUS-D1-CL', 'NULL', 't...; ^;[errorcode: 0] ... 2024-09-12 16:54:59 ERROR [expr.c:189/nx_expr_statement_execute()] assignment failed at line 61, character 70 in /etc/nxlog/nxlog.conf. statement execution has been aborted;[expr.c:90/nx_expr_statement_assignment_execute()] -;[expr.c:509/nx_expr_evaluate()] function 'convert' failed at line 61, character 69 in /etc/nxlog/nxlog.conf. expression evaluation has been aborted;[expr.c:279/nx_expr_eval_func()] -;[xm_charconv_funcproc_cb.c:283/nx_expr_func__convert()] -;[str.c:106/_nx_string_new_size()] oversized string (1688710), limit is 1048576 bytes 2024-09-12 16:54:59 DEBUG logdata missing or undef 'EventTime', setting to NULL 2024-09-12 16:54:59 DEBUG om_dbi SQL: INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Р В Р’В РВ<C2> 2024-09-12 16:54:59 ERROR [om_dbi.c:256/om_dbi_write()] -;[om_dbi.c:85/om_dbi_error()] om_dbi failed to execute SQL statement "INSERT INTO msglog.msglog (facility, severity, hostname, timestamp, application, message) VALUES ('USER', 'INFO', 'SERVER1-D1-CL', 'NULL', 'tp_msg', 'Р В Р’ВРР<E2><80> ...As you can see, logdata variable $EventTime just disappear and $Message get corrupted after some retries (note: om_dbi retries crazy fast - about 50000 time every few seconds). If I change om_dbi to simple om_file then all works fine. Another note: this configuration worked fine some time since initial setup, but now error appear every time (maybe postgresql query time affects it?)  

Please give me some advice. When I used nxlog to receive DHCP server logs, I found that the logs were missing and the nxlog logs showed error messages. INFO inode changed for 'E:\DHCP\DhcpSrvLog-Fri.log' (56→56): reopening possibly rotated file  

We currently have NXLog running on Windows 2019 with the file integrity module that monitors files in sub directories under a main directly.  I.E.  Say we have about 20 subdirectories for files under a directory called c:\code. If any file is changed/deleted/added under that directory, an alert message is sent out via the OM_UDP module to our siem. The problem I am looking to resolve, is that I will be copying over about 10 new directories with hundreds of files in each directory, and I don't want NXLog to generate hundreds of alerts because it found new files and directories.  Is there a way to “reset” or “refresh” the baseline after the new files/folders are copied over so it will know those files should be there?

Is there any ETA on Debian 12 support for NXLog Community Edition?

I havent been able to download the files for any of the community edition agents. https://nxlog.co/downloads/nxlog-ce#nxlog-community-edition , select the OS version and click Download.  5 sec later I get a pop up stating “Undefined”.  I have tried on different Browsers, platforms, and workstations and there is no change in the behavior.