1
response

Hey Guys, we are trying to send out our Windows DHCP logs which requires us to massage the log before we can send it out to the remote server.

Afraid I am not able to get the CSV Parser running, so far only the drop logic works, the intended output $Message does not come up in our output

` Panic Soft #NoFreeOnExit TRUE

define ROOT     C:\Program Files\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf\nxlog.d
define LOGDIR   %ROOT%\data


define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _charconv>
    Module      xm_charconv
    AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32, UCS-2LE
</Extension>

<Extension dhcp_csv_parser>
    Module      xm_csv
    Fields      ID, Date, Time, Description, IPAddress, Hostname, MACAddress, \
                UserName, TransactionID, QResult, ProbationTime, CorrelationID, \
                DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \
                UserClassASCII, RelayAgentInformation, DnsRegError
</Extension>


<Extension _exec>
    Module                       xm_exec
</Extension>


<Input ironnet_DHCPlogs>
    Module                       im_file
    File                         "C:\Windows\System32\dhcp\DhcpSrvLog-*.log"
    ReadFromLast                 TRUE
    SavePos                      TRUE
    <Exec>
        # Only process lines that begin with an event ID
        if $raw_event =~ /^\d+,/
        {

            dhcp_csv_parser->parse_csv();
            $QResult = integer($QResult);
            if $QResult == 0 $QMessage = "NoQuarantine";
            else if $QResult == 1 $QMessage = "Quarantine";
            else if $QResult == 2 $QMessage = "Drop Packet";
            else if $QResult == 3 $QMessage = "Probation";
            else if $QResult == 6 $QMessage = "No Quarantine Information";

            $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S');
            $ID = integer($ID);
            $ColonMAC = $MACAddress;
            if $ColonMAC =~ /^([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})$/ {
                $ColonMAC = $1 + ":" + $2 + ":" + $3 + ":" + $4 + ":" + $5 + ":" + $6;
            } 
            # DHCP Event IDs
            if $ID == 0 $Message = "The log was started.";
            else if $ID == 1 $Message = "The log was stopped.";
            else if $ID == 2 $Message = "The log was temporarily paused due to low disk space.";
            else if ($ID >= 10 and $ID <= 16) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName + " |" + $ColonMac;
            else if $ID == 17 drop();
            else if $ID == 18 drop();
            else if ($ID >= 20 and $ID <= 23) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName;
            else if $ID == 24 $Message = "IP address cleanup operation has began.";
            else if $ID == 25 $Message = "IP address cleanup statistics.";
            else if $ID == 30 drop();
            else if $ID == 31 drop();
            else if $ID == 32 drop();
            else if $ID == 33 $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName;
            else if $ID == 34 drop();
            else if $ID == 35 drop();
            else if $ID == 36 drop();
            else if ($ID >= 50 and $ID < 1000)
                $Message = "Codes above 50 are used for Rogue Server Detection " +
                           "information.";

            else drop();
        }
    </Exec>
</Input>

<Output out_ionnet_datacollector2> 
    Module                       om_udp
    Host                         172.16.10.42
    Port                         514
    OutputType                   LineBased
</Output>


<Route ionnet_DHCPlogs>
    Path        ionnet_DHCPlogs => out_ionnet_datacollector2
</Route> 

`

Input
30,04/05/22,16:04:58,DNS Update Request,10.41.22.51,P21023LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0 11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0 32,04/05/22,16:04:58,DNS Update Successful,10.41.22.51,P2103LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0

Output

11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0

AskedApril 8, 2022 - 4:13pm

Answer (1)

Hello,

Thanks for reaching out to us.

It appears that, in your input module, you run a csv parse procedure to read the $raw_event message data and create fields within NXLog. After that, you do your massaging. However, following the massaging, you have not specified NXLog to write those field data back to $raw_event again.

The $raw_event field is the field which is actually shipped when the event leaves the output. In order to write your fields to it, you can manually set it with $raw_event = $field1 + $field2, etc... or use one of our extension modules to translate them into the appropriate formats. Examples would be to_csv(); or to_json();.

I hope this helps.

Regards, Nathan L.