Log fan-out, also called dual-shipping, sends one copy of each event to two or more SIEMs from a single collection point. Teams do it during SIEM migrations, when an MSSP and an internal SOC both need raw events, for compliance archives, and for regional SIEMs with different regulatory scope. Do the split at the agent or at a relay. A second forwarder on every host doubles your configuration work and gives you two event counts that do not match. This article shows you how to configure fan-out in NXLog Agent, how to keep one destination’s outage from stalling the other, and how to prove both SIEMs received the same events.

What is log fan-out to multiple SIEMs?

Fan-out means the collector emits the same event to several destinations in parallel. Dual-shipping is the two-destination case. Some teams call it mirroring or teeing. The terms overlap, and this article uses them interchangeably when the destinations are SIEMs.

Most teams start with two agents per host: a Splunk Universal Forwarder next to the Azure Monitor Agent, for example. Two agents mean two configuration lifecycles, two sets of certificates to rotate, two CPU and memory footprints, and two event counts that differ because each agent reads the source at a slightly different moment.

NXLog Agent uses a different model. Its configuration separates inputs from outputs so you can route a single input to multiple outputs, and the same input can feed several routes at once. The agent reads each source once while running as a single process on the host, and sends the events to as many destinations as you need.

When does dual-shipping make sense?

Four situations come up repeatedly. Each one prevents a specific failure.

SIEM migration

Your detections live in the old SIEM. Rebuilding them in the new one takes months, and rules that look correct on paper fail in production. Dual-shipping keeps the old SIEM protecting you while you validate the new one against live data.

MSSP or MDR plus an internal SIEM

The provider needs raw events for their detections. Your team needs its own copy for hunting, compliance, and the day you change providers.

Compliance or cold archive

The SIEM holds hot data for detection. Object storage or a cheaper log store holds the multi-year retention copy. This is the scenario most likely to become permanent, and that is fine, because it splits data by purpose instead of duplicating cost.

Regional or business-unit SIEMs

The same source feeds a group SOC and a regional SOC that operates under different data-residency rules.

In Microsoft’s guidance for deploying Sentinel side-by-side with an existing SIEM, the recommended architecture runs both SIEMs just long enough to complete the migration. The guidance lists the drawbacks of a longer overlap: analytics split across different databases, investigations split for multi-environment incidents, higher staff and infrastructure costs, and analysts who need to know both tools. Microsoft’s preferred side-by-side method is to forward alerts from the legacy SIEM, not raw logs, because that avoids paying for ingestion twice. Dual-shipping raw events is the heavier option. Choose it when you need to validate the new SIEM’s detections against live data, filter what you send (Pattern 2 below), and set a cutover date before you turn it on.

The SANS 2025 SOC Survey (Christopher Crowley, July 2025, respondents from 57 countries) found that 42% of SOCs send all incoming data into a SIEM, often without a retrieval or management plan. The report describes storing everything as easy to justify today and hard to pay for tomorrow. Doubling that data to a second SIEM without filtering doubles the bill.

Four ways to fan out logs, and what each one costs

Approach Where the split happens Per-destination transform Independent buffering Ops overhead

Two agents per host

On every endpoint

Yes, but in two config systems

Yes

Highest: two agents, two lifecycles, mismatched counts

One agent, multiple outputs

On every endpoint, inside one process

Yes

Yes, per output queue

Low once the configuration is centrally managed

Relay or collector tier

On a small number of relay hosts

Yes

Yes

Low for endpoints; the relay tier needs sizing and HA

SIEM A forwards raw logs to SIEM B

Inside the first SIEM

Limited to what SIEM A exposes

No

Low to set up, high in cost and lock-in

Avoid the last row. Microsoft’s side-by-side guidance lists both "send logs from a legacy SIEM to Sentinel" and "send Sentinel logs to your legacy SIEM" among the configurations it does not recommend, because you pay for ingestion in both products and add architectural complexity. Forwarding alerts between SIEMs is a different case: that is Microsoft’s recommended side-by-side method.

Our recommendation: fan out at the agent for endpoints and servers, and at a relay tier for syslog and network sources that cannot run an agent. The rest of this article shows both.

How do you configure fan-out in NXLog Agent?

The examples below use Windows Security events shipped to Splunk over HTTP Event Collector and to Microsoft Sentinel over the Logs Ingestion API. The same structure applies to any of NXLog Agent’s output modules. Each pattern builds on the one before it, and every directive is taken from the NXLog Agent v6.16 reference documentation or our integration guides.

Pattern 1: one route, multiple outputs

In NXLog Agent, a route’s Path connects inputs on the left to outputs on the right. Outputs are comma-separated because they run in parallel; processors are separated by arrows because they run in series. A single route can therefore send every event from one input module to several output modules, and that one line is the fan-out.

Each output module also has its own Exec block, so it can process and rewrite its copy of the record independently of the other outputs. That matters because two SIEMs rarely accept the same record shape. Splunk wants the HEC envelope. Sentinel’s Logs Ingestion API expects a JSON object whose field names match the DCR stream’s columns exactly. Column names are case-sensitive, and the Microsoft Azure Logs Ingestion output module reference warns that records which do not match the expected format are silently discarded.

nxlog.conf
<Extension json>
    Module            xm_json
</Extension>

<Extension hec_envelope>
    Module            xm_rewrite
    Keep              time, host, source, sourcetype, event
</Extension>

<Input windows_security>
    Module            im_msvistalog
    Channel           Security
</Input>

<Output splunk_hec>
    Module            om_http (1)
    URL               https://splunk.example.com:8088/services/collector/event
    AddHeader         Authorization: Splunk <HEC_TOKEN>
    HTTPSCAFile       %CERTDIR%/splunk-ca.pem
    BatchMode         multiline
    Compression       gzip
    <Exec> (2)
        $event = to_json();
        $sourcetype = "_json";
        if $Hostname { $host = $Hostname; }
        $time = string(integer($EventTime));
        $time =~ /^(?<sec>\d+)(?<ms>\d{6})$/;
        $time = $sec + "." + $ms;
        hec_envelope->process();
        to_json();
    </Exec>
</Output>

<Output sentinel>
    Module            om_azuremonitor (3)
    ClientId          <ENTRA_APP_ID>
    TenantId          <ENTRA_TENANT_ID>
    ClientSecret      <ENTRA_CLIENT_SECRET>
    URL               https://<DCE_NAME>.ingest.monitor.azure.com
    DcrImmutableId    <DCR_IMMUTABLE_ID>
    StreamName        Custom-WindowsSecurity_CL
    <Exec>
        $out_template = ( (4)
            'TimeGenerated' => $EventTime,
            'Computer'      => $Hostname,
            'Channel'       => $Channel,
            'EventID'       => $EventID,
            'EventRecordId' => $RecordNumber,
            'EventData'     => $Message
        );
        $raw_event = json->to_json($out_template); (5)
    </Exec>
</Output>

<Route fanout>
    Path              windows_security => splunk_hec, sentinel (6)
</Route>
1 The HTTP(s) output module posts to the HEC event endpoint.
2 Wraps the record in the HEC event envelope.
3 The Microsoft Azure Logs Ingestion output module authenticates with a Microsoft Entra application and writes through a data collection endpoint (DCE) to a data collection rule (DCR).
4 Builds the record the DCR stream expects.
5 Converts the record to JSON and writes it to $raw_event, which is what the output sends.
6 Sends every event from the input to both outputs, and each output shapes its own copy.

The Splunk output follows our Splunk integration guide. Splunk’s own HTTP Event Collector documentation covers creating the token and enabling indexer acknowledgment.

For the Sentinel output, our Sentinel integration guide walks through the app registration, DCE, and DCR setup, ships a DCR template for Windows events with the full field list, and includes a certificate-based authentication variant if your security policy prohibits client secrets.

Pattern 2: send a filtered subset to the second SIEM

Dual-shipping everything doubles ingestion cost. Often the second SIEM only needs a slice: authentication events during a Sentinel evaluation, or privileged-account activity for the MSSP. The drop() procedure discards the record being processed. Because it runs inside the output’s Exec block, it only affects that destination. Our how-to on prioritizing data forwarding uses the same technique to send different severities to different outputs.

nxlog.conf
<Output sentinel>
    Module    om_azuremonitor
    # ... authentication directives as above ...
    <Exec>
        if ($EventID == 4624 or $EventID == 4625 or
            $EventID == 4672 or $EventID == 4720 or
            $EventID == 4728)
        {
            $out_template = (
                'TimeGenerated' => $EventTime,
                'Computer'      => $Hostname,
                'Channel'       => $Channel,
                'EventID'       => $EventID,
                'EventRecordId' => $RecordNumber,
                'EventData'     => $Message
            );
            $raw_event = json->to_json($out_template);
        }
        else
        {
            drop(); (1)
        }
    </Exec>
</Output>
1 Drops all other events, so they never reach Sentinel.

Splunk still gets the full Security channel. Sentinel gets the events you would build your first detections on: successful and failed logons (4624, 4625), special privileges assigned to a new logon (4672), a user account created (4720), and a member added to a security-enabled global group (4728). When you are ready to widen the scope, you change one condition and push the configuration.

Pattern 3: keep one destination’s outage from stalling the other

With one output, NXLog Agent’s flow control does what you want: when the destination blocks, the agent suspends the upstream module until the output catches up, and nothing is lost from a pausable source. With two outputs sharing an input, the same mechanism means that when Sentinel’s ingestion endpoint returns errors for an hour, the input pauses and Splunk stops receiving too. The common module settings reference covers this case: in a route with several outputs, the whole route blocks when any one output blocks, unless you turn flow control off on the outputs where events may be discarded.

You have two options, and the source decides which one is right.

Option A: leave flow control on

For pausable sources such as files and Windows Event Log, nothing is lost. The agent stops reading, the source keeps accumulating, and when the slow SIEM recovers both destinations catch up. The cost is latency on the healthy SIEM for the length of the outage. If the detections in your production SIEM can tolerate that delay, this is the safer default, and it is the only option that rides out a long outage without loss.

Option B: turn flow control off on the output that may lose data, and give it a persistent queue

Set FlowControl FALSE on the second SIEM’s output only. The input keeps reading, the production SIEM keeps receiving in real time, and the blocked output buffers to its queue limit, then discards until it recovers. Flow control on the production output stays on, so if Splunk blocks, the input still pauses and you still lose nothing there. NXLog’s buffering how-to shows the same FlowControl pattern for a route with two outputs.

nxlog.conf
<Input windows_security> (1)
    Module             im_msvistalog
    Channel            Security
</Input>

<Output splunk_hec>
    Module             om_http
    # ... directives as above; flow control stays on ...
</Output>

<Output sentinel>
    Module             om_azuremonitor
    # ... directives as above ...
    FlowControl        FALSE (2)
    LogqueueSize       16777216 (3)
    PersistLogqueue    TRUE
</Output>

<Route fanout>
    Path               windows_security => splunk_hec, sentinel
</Route>
1 Flow control stays on, so the input pauses if Splunk blocks.
2 Stops this output from blocking the route when Sentinel is unreachable.
3 Sets a 16 MiB disk-backed queue. Size it from your own event rate.

LogqueueSize is a soft limit in bytes on the output’s queue. The reference gives a 2 MiB default for an in-memory queue and a smaller one for a disk-based queue. Size it from your own numbers: event rate × outage length × average event size. A Windows Security event serialized as JSON runs roughly 1 to 2 KB, so the 16 MiB queue above holds about 8,000 to 16,000 events. At 200 events per second that is around a minute; on a domain controller at 1,000 events per second, closer to 10 seconds. Treat Option B as protection against short interruptions such as throttling or a brief network drop. For longer outages, use Option A, or accept that the second SIEM will miss what the queue cannot hold.

PersistLogqueue TRUE keeps the queue on disk while the agent runs. An in-memory queue is written to disk on a clean shutdown anyway, so the directive buys you protection against a crash or power loss, at the cost of more disk I/O. Add SyncLogqueue TRUE only if you can accept a disk sync per record.

One variant to know about. Setting FlowControl FALSE on the input instead of the output also keeps the route moving, but it removes the protection from both destinations: whichever output blocks, that output discards once its queue fills. The reference reserves this for sources that should never be suspended, such as the UDP and Linux Audit System input modules, and the /dev/log socket, where pausing the input would drop packets or block the calling program. For Windows Event Log and files, put the directive on the output.

Pattern 4: fan out at a relay tier

Firewalls, switches, and appliances cannot run an agent, so they send syslog to a relay, and the relay does the fan-out. Endpoint agents can ship to the same relay. They stay simple: one output, one TLS certificate, and no knowledge of which SIEMs exist downstream.

nxlog.conf
<Extension json>
    Module         xm_json
</Extension>

<Extension syslog>
    Module         xm_syslog
</Extension>

<Input network_syslog> (1)
    Module         im_udp
    ListenAddr     0.0.0.0:514
    FlowControl    FALSE (2)
    Exec           parse_syslog();
</Input>

<Input agents_a>
    Module         im_tcp
    ListenAddr     0.0.0.0:1514
    ReusePort      TRUE (3)
    Exec           parse_json();
</Input>

<Input agents_b>
    Module         im_tcp
    ListenAddr     0.0.0.0:1514
    ReusePort      TRUE
    Exec           parse_json();
</Input>

# splunk_hec and sentinel outputs as defined above

<Route relay_fanout>
    Path           network_syslog, agents_a, agents_b => splunk_hec, sentinel
</Route>
1 Receives syslog from network devices.
2 Turns flow control off because UDP input cannot be paused.
3 Lets the two identical listeners for endpoint agents share port 1514, so the operating system spreads connections across threads.

In practice, give the network syslog its own Sentinel output and DCR stream, since its fields differ from the Windows events the Pattern 1 template expects.

The ReusePort directive lets several input instances receive on the same port, which matters because each connection is bound to a single input thread. Our guide on distributing NXLog Agent load explains why a single hot source can otherwise cap throughput at one thread’s saturation point. For production, swap the TCP input module for the TLS/SSL input module on port 6514 so the relay terminates TLS from the endpoints.

Rolling out fan-out across a fleet with NXLog Platform

The configurations above are per-agent files. That is fine for a handful of hosts. Across a fleet, adding the second SIEM has to be one change, deployed once.

NXLog Platform manages agent configuration centrally. A configuration is defined once and assigned to as many agents as you need. Every save creates a new version, and you can compare two versions and roll back from the version history. In practice: duplicate the production configuration, add the Sentinel output and list it on the route’s Path, assign the new configuration to a pilot group, watch the counts, then update the rest of the fleet. On cutover day, remove the Splunk output the same way, with the previous version one click away if you need it.

The configuration builder has a text view, so you can paste the examples from this article and switch back to the visual editor. If you need to automate, the Agent Management API lets you create and update configurations with POST requests, so you can keep the fan-out configuration in version control next to your detection content.

How do you verify both SIEMs received the same events?

During an incident review, someone will ask whether both SIEMs hold the same events. Four checks answer that, in the order we would run them.

Tag each event at the agent

Add a field the source guarantees is unique, so the same event is traceable in both SIEMs. For Windows Event Log, the record number does the job.

nxlog.conf
<Input windows_security>
    Module     im_msvistalog
    Channel    Security
    Exec       $FanoutId = $Hostname + ":" + $Channel + ":" + string($RecordNumber);
</Input>

For the Sentinel branch, map $FanoutId into a column that exists in your DCR stream. A field the DCR does not declare does not reach the table.

Count and diff

Query events per host per hour in each SIEM, keyed on FanoutId. During a migration this is the daily report that tells you whether the new SIEM is trustworthy yet. A persistent gap on one host usually means the destination’s parser is rejecting records. A transport problem shows up as a gap across all hosts at once.

Watch the agent’s own logs

NXLog Agent logs a warning to its internal log when an output’s queue fills. Route those messages somewhere you will see them with the Internal input module.

Test the outage before it happens

The Blocker output module simulates a blocked destination. Stand in for Sentinel with the Blocker output module, generate load, and confirm Splunk keeps receiving with whichever option you chose in Pattern 3. Then check the queue size you picked against the event rate you generated.

Common mistakes when dual-shipping logs

Most dual-shipping problems come from treating the second destination as a copy of the first, instead of a separate integration with its own format, certificates, and failure modes.

Normalizing once for two schemas

Splunk’s _json sourcetype and Sentinel’s DCR expect different envelopes and field names. Shape per destination, as in Pattern 1.

Forgetting the second destination in certificate rotation

Two outputs mean two CA files and two expiry dates. Put both on the same calendar.

No alert on queue high-water marks

If you chose Option B in Pattern 3, a full queue means silent loss at the second SIEM. Alert on the agent’s internal warnings before that happens.

Turning flow control off on the input

That removes loss protection from both destinations. Put FlowControl FALSE on the output you can afford to lose.

Timestamp drift between destinations

One SIEM keyed on EventTime, the other on ingest time, and your diff report will not reconcile. Map the event time explicitly for both.

Letting side-by-side become permanent by accident

Microsoft lists higher staff and infrastructure costs among the drawbacks of a long side-by-side period. If the second destination is an archive, that is a design. If it is a SIEM you meant to decommission, put the date in the project plan.

Where to go from here

NXLog Agent gives you the fan-out in a single Path line, per-destination shaping in Exec blocks, and explicit control over what happens when one SIEM goes dark. NXLog Platform lets you roll that configuration out, and back, across the fleet from one place.

If you are planning a migration or adding a second destination, we are happy to review your route design with you. Talk to us, or start a free trial of NXLog Platform with your own agents.

FAQ

Does dual-shipping double my SIEM cost?

Only if you send everything to both. Filtering at the agent, as in Pattern 2, lets you send the full stream to one SIEM and a targeted subset to the other, so ingestion-based pricing scales with what each SIEM needs. Running two full SIEMs for an extended period does roughly double ingestion and storage cost, which is why we recommend a defined end date for migrations, in line with Microsoft’s recommended side-by-side architecture.

Can NXLog Agent send the same log to Splunk and Microsoft Sentinel at the same time?

Yes. A single <Route> with Path input => splunk_output, sentinel_output sends every event to both. Splunk receives via the HTTP(s) output module to the HTTP Event Collector, and Sentinel via the Microsoft Azure Logs Ingestion output module to the Logs Ingestion API. The source is read once.

What happens if one SIEM is unreachable?

That depends on flow control. With the default settings, the agent pauses the shared input until the unreachable SIEM recovers, so both destinations lag but nothing is lost from pausable sources. With FlowControl FALSE on the unreachable SIEM’s output, the other SIEM keeps receiving in real time and the blocked output buffers to its queue limit, then discards until it recovers. Pattern 3 covers both configurations.

Is fan-out the same as a telemetry pipeline?

Fan-out is one capability a telemetry pipeline provides. A dedicated pipeline product adds a separate tier of infrastructure to do routing, filtering, and transformation. NXLog Agent performs those functions on the endpoint or on a relay you already run, so for many teams the fan-out use case does not need a new tier.

Should the fan-out happen on the endpoint or on a relay?

On the endpoint for servers and workstations that run an agent: one process, one read of the source, no extra hop. On a relay for network devices and appliances that cannot run an agent, and for environments where endpoints may only talk to one internal address. Many deployments use both.

NXLog Platform is an on-premises solution for centralized log management with
versatile processing forming the backbone of security monitoring.

With our industry-leading expertise in log collection and agent management, we comprehensively
address your security log-related tasks, including collection, parsing, processing, enrichment, storage, management, and analytics.

Start free Contact us
Share
Related Posts