Commit Graph

29 Commits

Author SHA1 Message Date
gezimbll
3ebf3007a5 added startdelay config option for ers readers 2025-01-29 14:44:28 +01:00
ionutboangiu
1c73e42307 Revise ERs event exporting
Make use of the previously added ees_success_ids and ees_failed_ids
configuration fields.

Remove Processed opts and everything related to them since they should
not be used anymore.

Fixed test compilation errors caused by the change.
2024-09-05 20:46:40 +02:00
arberkatellari
c7221f0d1e Add reader id within the *vars in ERs 2024-08-12 18:34:50 +02:00
ionutboangiu
d0a435aa6d Migrate to new jetstream API
Upgraded go.mod nats version due to an issue caused by version mismatch
between driver and server (uncertain).

Renamed function from getProcessOptions to getProcessedOptions.

## *NatsER.Serve

- Replaced ChanQueueSubscribe with QueueSubscribe for Core NATS consumer
to handle the message processing directly.

- Since QueueSubscribe is now used regardless of jetstream status, the
message handler has been assigned to a separate variable that can be
reused.

-  The message handler is now dealing with the message processing
directly, therefore the select case listening for the channel which is
feeding NATS messages can be removed together with the channel itself
and the select. Currently, the goroutine within Serve only has to block
until the rdrExit chan is closed.

- Moved the resource check inside the handler right before starting the
message processing goroutine.

## *NatsEE.parseOpts

- Renamed function from parseOpt to parseOpts.

- Handled the error coming from GetNatsOpts function.

## *NatsEE.Connect

- Updated function to return early in case of non-nil nats.Conn value
to reduce nesting.

## *NatsEE.ExportEvent

- Use defer to release resources and RUnlock.

## *NatsEE.Close

- Use defer to Unlock.

- Update function to return early in case of nil nats.Conn value to
reduce nesting.

## ees.GetNatsOpts

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

## ers.GetNatsOpts

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

Removed tab from commented natsJetStreamMaxWaitProcessed option
value in config_defaults.go under ers section.

Added integration test for ERs NATS.

Updated ees/ers implementation to use the jetstream package which
separates the jetstream context from Core NATS.

Removed the jsOpts fields from the NatsEE struct. We are now using the
jetStreamMaxWait option directly through a timeout context.

Added streamName option for NATS reader since it is now required to be
specified when creating a consumer (it is not inferred based on subject
anymore).

Updated nats ers integration tests.

Updated tests to also use the new jetstream package.

Updated tests to start the nats-server using their official driver
instead of using the std go exec package. time.Sleeps are now not
required anymore to wait for the server.

In test configurations for nats readers, made sure that natsStreamName
option is populated. It is now required for consumers to know where to
subscribe.
2023-09-28 20:48:39 +02:00
ionutboangiu
113e2a2bdf Replace interface{} with any 2023-05-31 10:22:28 +02:00
ionutboangiu
8620b23231 Implement EventExporterCfg constructor and get rid of boilerplate code 2023-03-08 19:12:48 +01:00
adi
a448fb2c7a INtegrated efs in logger/ees + ers changes 2022-07-30 08:19:08 +02:00
adi
86d2f1476a Attemtps config change 2022-07-30 08:19:08 +02:00
adi
341f33873b Improved attempts and faield posts dir cfg in action/ers 2022-07-19 17:15:42 +02:00
adi
8b14619fce Integrated efs 2022-07-13 19:24:50 +02:00
andronache98
ecdf5e54ea Changed headers 2022-03-09 14:14:30 +01:00
andronache98
9ec56f5155 Changed headers 2022-03-09 14:14:30 +01:00
ionutboangiu
8ce6982324 Remove <populated> return parameter from GetProcessOptions 2021-11-01 13:51:02 +02:00
ionutboangiu
316e9d6f32 Change type of ExportEvent opts to *EventExporterOpts and fix compilation errors 2021-11-01 13:51:02 +02:00
ionutboangiu
aa46b1eefd Fix compilation errors after making ers opts into structs 2021-11-01 13:51:02 +02:00
Trial97
a3ebbe38ec Added EeSV1 APIs back 2021-09-07 07:44:10 +02:00
Trial97
9aa153c0b0 Finished implementing all the exporters 2021-08-18 16:34:44 +02:00
Trial97
c28ac62968 Updated flatstore tests 2021-05-19 15:54:14 +02:00
Trial97
d868c0a5f0 Added partial reader 2021-05-19 15:54:14 +02:00
Trial97
f6863bad3e Updated flatsore ers 2021-05-06 14:00:37 +02:00
Trial97
45a11aa91e Updated s3 and sqs ers configs 2021-05-04 20:45:09 +02:00
Trial97
8aa7009d36 Updated AttributesSv1 APIs 2021-04-13 13:35:03 +02:00
Trial97
c13a5cf790 Finished implementing the DataNode as a replacement 2021-03-26 13:46:25 +01:00
Trial97
5353b95e43 Updated go version and replaced ioutil package 2021-03-18 16:46:55 +01:00
TeoV
2bfb234804 Update integration test in agents 2021-01-15 09:53:55 +01:00
TeoV
906baa051a Build success 2021-01-14 10:30:35 +01:00
Trial97
fc9eb41e7e Added AMQPv1 Event Reader 2020-10-20 11:09:16 +02:00
Trial97
51abe1c7c8 Added SQS Event Reader 2020-10-20 11:09:16 +02:00
Trial97
49e423d21a Added S3 Event Reader 2020-10-20 11:09:16 +02:00