- use kraft instead of zookeeper
- add handlers in case of cfg changes
- create a separate user for the kafka service
- bump kafka version
- make the role more configurable
Previously, msg.Ack(true) was used, which is mostly used for batch
processing. It mistakenly acknowledged all previously unacknowledged
messages, causing errors from the AMQP server. Now, messages are
acknowledged individually after each one is processed.
Messages that ERs failed to process are now rejected and requeued
for future processing attempts.
The reader is now closed immediately if the message delivery
channel closes. Therefore, it prevents an endless loop by avoiding
continuous consumption from empty or closed channels.
Addresses: #4146
Upgraded go.mod nats version due to an issue caused by version mismatch
between driver and server (uncertain).
Renamed function from getProcessOptions to getProcessedOptions.
## *NatsER.Serve
- Replaced ChanQueueSubscribe with QueueSubscribe for Core NATS consumer
to handle the message processing directly.
- Since QueueSubscribe is now used regardless of jetstream status, the
message handler has been assigned to a separate variable that can be
reused.
- The message handler is now dealing with the message processing
directly, therefore the select case listening for the channel which is
feeding NATS messages can be removed together with the channel itself
and the select. Currently, the goroutine within Serve only has to block
until the rdrExit chan is closed.
- Moved the resource check inside the handler right before starting the
message processing goroutine.
## *NatsEE.parseOpts
- Renamed function from parseOpt to parseOpts.
- Handled the error coming from GetNatsOpts function.
## *NatsEE.Connect
- Updated function to return early in case of non-nil nats.Conn value
to reduce nesting.
## *NatsEE.ExportEvent
- Use defer to release resources and RUnlock.
## *NatsEE.Close
- Use defer to Unlock.
- Update function to return early in case of nil nats.Conn value to
reduce nesting.
## ees.GetNatsOpts
- Chose switch over if else when parsing client certificate and keys
opts.
- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.
## ers.GetNatsOpts
- Chose switch over if else when parsing client certificate and keys
opts.
- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.
Removed tab from commented natsJetStreamMaxWaitProcessed option
value in config_defaults.go under ers section.
Added integration test for ERs NATS.
Updated ees/ers implementation to use the jetstream package which
separates the jetstream context from Core NATS.
Removed the jsOpts fields from the NatsEE struct. We are now using the
jetStreamMaxWait option directly through a timeout context.
Added streamName option for NATS reader since it is now required to be
specified when creating a consumer (it is not inferred based on subject
anymore).
Updated nats ers integration tests.
Updated tests to also use the new jetstream package.
Updated tests to start the nats-server using their official driver
instead of using the std go exec package. time.Sleeps are now not
required anymore to wait for the server.
In test configurations for nats readers, made sure that natsStreamName
option is populated. It is now required for consumers to know where to
subscribe.
