Using om_redis with mulitple output servers

#1 davidatpinger 7 years ago

Hey!

I have a bunch of log data that I'd like to push to a set of redis instances that are used as a queue. In an ideal world, I could use a config like this:

<Output redis-out>
Command LPUSH
Host server1,server2,server3
</Output>

The desired result is that nxlog pushes sequential lines to each of the servers in turn - no duplication, just load distribution.

Is there presently a way to do this? I'm thinking I'm likely to end up running through a hardware load-balancer, but it sure would be nice to just do this in nxlog.

Thanks!

Permalink

#2 b0ti Nxlog ✓ 7 years ago

#1 davidatpinger

Hey! I have a bunch of log data that I'd like to push to a set of redis instances that are used as a queue. In an ideal world, I could use a config like this: <Output redis-out> Command LPUSH Host server1,server2,server3 </Output> The desired result is that nxlog pushes sequential lines to each of the servers in turn - no duplication, just load distribution. Is there presently a way to do this? I'm thinking I'm likely to end up running through a hardware load-balancer, but it sure would be nice to just do this in nxlog. Thanks!

There are plans to add failover support to the various networking modules but load bancing is slightly out of scope. The problem with the latter is that it's hard to do right and there are too many parameters.

Currently your option is to use third-party tools. This does not need to be hardware, you can use haproxy and similar software based solutions.