gzipped content to a socket | Log collection solutions

#1 davidatpinger 6 years ago

Hey!

I can think of several ways to solve this, but I'm looking to see if anyone has a better idea. I'm getting some logs that are uploaded as a gzipped file. I need to uncompress the file and then send the log content to several destinations, but for this purpose, let's assume that I'm using om_tcp as the destination. I'm considering a couple of options, but has anyone already done something like this or have a brilliant idea for handling it?

Thanks!

Permalink

#4 b0ti Nxlog ✓ 6 years ago (Last updated 6 years ago )

#1 davidatpinger

Hey! I can think of several ways to solve this, but I'm looking to see if anyone has a better idea. I'm getting some logs that are uploaded as a gzipped file. I need to uncompress the file and then send the log content to several destinations, but for this purpose, let's assume that I'm using om_tcp as the destination. I'm considering a couple of options, but has anyone already done something like this or have a brilliant idea for handling it? Thanks!

Built-in compression support is on the roadmap. Unfortunately this didn't make it into the latest NXLog EE v4.0 release.

Would something like the following work? You can use a script via im_exec that uncompresses the files:

#!/bin/sh

while true; do
 for i in *.log.gz; do
   zcat $i; 
   mv $i /processed/;
 done
 sleep 1;
done