DHCP Nxlog Customisation

Tags:

#1 SecuronixS

Hey Guys, we are trying to send out our Windows DHCP logs which requires us to massage the log before we can send it out to the remote server.

Afraid I am not able to get the CSV Parser running, so far only the drop logic works, the intended output $Message does not come up in our output

` Panic Soft #NoFreeOnExit TRUE

define ROOT     C:\Program Files\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf\nxlog.d
define LOGDIR   %ROOT%\data


define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _charconv>
    Module      xm_charconv
    AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32, UCS-2LE
</Extension>

<Extension dhcp_csv_parser>
    Module      xm_csv
    Fields      ID, Date, Time, Description, IPAddress, Hostname, MACAddress, \
                UserName, TransactionID, QResult, ProbationTime, CorrelationID, \
                DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \
                UserClassASCII, RelayAgentInformation, DnsRegError
</Extension>


<Extension _exec>
	Module                       xm_exec
</Extension>


<Input ironnet_DHCPlogs>
	Module                       im_file
	File                         "C:\Windows\System32\dhcp\DhcpSrvLog-*.log"
	ReadFromLast                 TRUE
	SavePos                      TRUE
    <Exec>
        # Only process lines that begin with an event ID
        if $raw_event =~ /^\d+,/
        {
			
            dhcp_csv_parser->parse_csv();
            $QResult = integer($QResult);
            if $QResult == 0 $QMessage = "NoQuarantine";
            else if $QResult == 1 $QMessage = "Quarantine";
            else if $QResult == 2 $QMessage = "Drop Packet";
            else if $QResult == 3 $QMessage = "Probation";
            else if $QResult == 6 $QMessage = "No Quarantine Information";

            $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S');
            $ID = integer($ID);
			$ColonMAC = $MACAddress;
			if $ColonMAC =~ /^([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})$/ {
				$ColonMAC = $1 + ":" + $2 + ":" + $3 + ":" + $4 + ":" + $5 + ":" + $6;
			} 
            # DHCP Event IDs
            if $ID == 0 $Message = "The log was started.";
            else if $ID == 1 $Message = "The log was stopped.";
            else if $ID == 2 $Message = "The log was temporarily paused due to low disk space.";
            else if ($ID >= 10 and $ID <= 16) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName + " |" + $ColonMac;
            else if $ID == 17 drop();
            else if $ID == 18 drop();
            else if ($ID >= 20 and $ID <= 23) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName;
            else if $ID == 24 $Message = "IP address cleanup operation has began.";
            else if $ID == 25 $Message = "IP address cleanup statistics.";
            else if $ID == 30 drop();
            else if $ID == 31 drop();
            else if $ID == 32 drop();
            else if $ID == 33 $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName;
            else if $ID == 34 drop();
            else if $ID == 35 drop();
            else if $ID == 36 drop();
            else if ($ID >= 50 and $ID < 1000)
                $Message = "Codes above 50 are used for Rogue Server Detection " +
                           "information.";
           
            else drop();
        }
    </Exec>
</Input>

<Output out_ionnet_datacollector2> 
	Module                       om_udp
	Host                         172.16.10.42
	Port                         514
	OutputType                   LineBased
</Output>


<Route ionnet_DHCPlogs>
	Path		ionnet_DHCPlogs => out_ionnet_datacollector2
</Route> 

`

Input
30,04/05/22,16:04:58,DNS Update Request,10.41.22.51,P21023LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0 11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0 32,04/05/22,16:04:58,DNS Update Successful,10.41.22.51,P2103LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0

Output

11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0

#2 NathanDeactivated Nxlog ✓
#1 SecuronixS
Hey Guys, we are trying to send out our Windows DHCP logs which requires us to massage the log before we can send it out to the remote server. Afraid I am not able to get the CSV Parser running, so far only the drop logic works, the intended output $Message does not come up in our output ` Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf\nxlog.d define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32, UCS-2LE </Extension> <Extension dhcp_csv_parser> Module xm_csv Fields ID, Date, Time, Description, IPAddress, Hostname, MACAddress, \ UserName, TransactionID, QResult, ProbationTime, CorrelationID, \ DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \ UserClassASCII, RelayAgentInformation, DnsRegError </Extension> <Extension _exec> Module xm_exec </Extension> <Input ironnet_DHCPlogs> Module im_file File "C:\Windows\System32\dhcp\DhcpSrvLog-*.log" ReadFromLast TRUE SavePos TRUE <Exec> # Only process lines that begin with an event ID if $raw_event =~ /^\d+,/ { dhcp_csv_parser->parse_csv(); $QResult = integer($QResult); if $QResult == 0 $QMessage = "NoQuarantine"; else if $QResult == 1 $QMessage = "Quarantine"; else if $QResult == 2 $QMessage = "Drop Packet"; else if $QResult == 3 $QMessage = "Probation"; else if $QResult == 6 $QMessage = "No Quarantine Information"; $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S'); $ID = integer($ID); $ColonMAC = $MACAddress; if $ColonMAC =~ /^([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})$/ { $ColonMAC = $1 + ":" + $2 + ":" + $3 + ":" + $4 + ":" + $5 + ":" + $6; } # DHCP Event IDs if $ID == 0 $Message = "The log was started."; else if $ID == 1 $Message = "The log was stopped."; else if $ID == 2 $Message = "The log was temporarily paused due to low disk space."; else if ($ID >= 10 and $ID <= 16) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName + " |" + $ColonMac; else if $ID == 17 drop(); else if $ID == 18 drop(); else if ($ID >= 20 and $ID <= 23) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 24 $Message = "IP address cleanup operation has began."; else if $ID == 25 $Message = "IP address cleanup statistics."; else if $ID == 30 drop(); else if $ID == 31 drop(); else if $ID == 32 drop(); else if $ID == 33 $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 34 drop(); else if $ID == 35 drop(); else if $ID == 36 drop(); else if ($ID >= 50 and $ID < 1000) $Message = "Codes above 50 are used for Rogue Server Detection " + "information."; else drop(); } </Exec> </Input> <Output out_ionnet_datacollector2> Module om_udp Host 172.16.10.42 Port 514 OutputType LineBased </Output> <Route ionnet_DHCPlogs> Path ionnet_DHCPlogs => out_ionnet_datacollector2 </Route> ` Input 30,04/05/22,16:04:58,DNS Update Request,10.41.22.51,P21023LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0 11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0 32,04/05/22,16:04:58,DNS Update Successful,10.41.22.51,P2103LAB.HEALTH.LOCAL,,,0,6,,,,,,,,,0 Output 11,04/05/22,16:04:58,Renew,10.41.22.51,P2103LAB.HEALTH.LOCAL,98EECBDE1CDE,,179078305,0,,,,0x4D53465420352E30,MSFT 5.0,,,,0

Hello,

Thanks for reaching out to us.

It appears that, in your input module, you run a csv parse procedure to read the $raw_event message data and create fields within NXLog. After that, you do your massaging. However, following the massaging, you have not specified NXLog to write those field data back to $raw_event again.

The $raw_event field is the field which is actually shipped when the event leaves the output. In order to write your fields to it, you can manually set it with $raw_event = $field1 + $field2, etc... or use one of our extension modules to translate them into the appropriate formats. Examples would be to_csv(); or to_json();.

I hope this helps.

Regards, Nathan L.