Skip to content
JMS Bridge Docs
Search limited to the 3.3.6 version. Switch to the latest version for up-to-date results.

JMS Kafka Divert Limitations

This content is for the 3.3.6 version. Switch to the latest version for up-to-date documentation.

JMS Producer to Kafka Topic Divert Processing on Crash Flow

Following flow highlights limitation that asynchronous processing of divert in Kafka Streams based message journal can have upon crash of JMS Bridge node. Kafka Streams processing state is saved periodically (known as Commit) and upon crash, the state is restored from the last commit point. Due to this, messages can be re-produced to Kafka Topics after JMS Bridge node crash - causing duplication on Kafka Topics only.

Sequence diagram below illustrates a flow where:

  • Message 1 is sent by JMS Producer
  • Message 1 is ingested into Journal
  • Message 1 is Acked back to JMS Producer
  • Message 1 is diverted to Kafka
  • KStreams state committed
  • Messages 2,3 are ingested into Journal
  • Messages 2,3 are Acked back to JMS Producer
  • Messages 2,3 are diverted to Kafka
  • Message 4 is sent by JMS Producer
  • Message 4 is ingested into Journal
  • JMS Bridge crashes
  • Standby JMS Bridge takes over
  • KStreams state is restored from last commit point
  • Messages 2,3 are re-processed and diverted to Kafka again (causing duplicates on Kafka Topic)
  • Message 4 is diverted to Kafka
  • JMS Producer sends Message 4* again (retry as Ack was not returned from JMS Bridge due to crash even though message was persisted to Journal)
  • Message 4* is ingested into Journal again (retry - duplicate on JMS Queue / Topic)
  • Message 4* is Acked back to JMS Producer (retry)
  • Message 4* is diverted to Kafka (retry - duplicate on Kafka Topic)

The handling of Message 4 is consistent with Vanilla Artemis and guarantees in Sync send mode, but duplication caused by reprocessing divert for messages 2 and 3 are side effects of Kafka Streams processing and asynchronous state commit.

Kafka TopicKafka Divert ProcessorMessage JournalJMS BridgeJMS ProducerKafka TopicKafka Divert ProcessorMessage JournalJMS BridgeJMS ProducerSave point for Journal Processing - KStream CommitJMS Bridge CrashesFailover to Standby JMS BridgeKStreams committed state loadedSync Send 1Kafka Produce 1 (Ingest)Kafka Produce 1 AckJMS Send 1 AckApply Divert 1Kafka Produce 1Commit KStream StateSync Send 2Kafka Produce 2 (Ingest)Kafka Produce 2 AckJMS Send 2 AckSync Send 3Kafka Produce 3 (Ingest)Kafka Produce 3 AckJMS Send 3 AckApply Divert 2Kafka Produce 2Apply Divert 3Kafka Produce 3Sync Send 4Kafka Produce 4 (Ingest)Kafka Produce 4 AckApply Divert 2Kafka Produce 2Apply Divert 3Kafka Produce 3Sync Send 4* (retry as ACK not received)Kafka Produce 4* (Ingest) (retry)Kafka Produce 4* Ack (retry)JMS Send 4* Ack (retry)Apply Divert 4 (original)Kafka Produce 4 (original)Apply Divert 4* (retry)Kafka Produce 4* (retry)