42 Commits

Author SHA1 Message Date
ionutboangiu
8037076ded ers: remove redundant concurrency channel init 2025-11-20 11:35:29 +01:00
arberkatellari
119481b3d1 Update ERs SQL functionalities 2025-10-13 09:58:45 +02:00
ionutboangiu
e05be683ae change license from GPLv3 to AGPLv3 2025-10-13 09:57:41 +02:00
gezimbll
3ebf3007a5 added startdelay config option for ers readers 2025-01-29 14:44:28 +01:00
ionutboangiu
1c73e42307 Revise ERs event exporting
Make use of the previously added ees_success_ids and ees_failed_ids
configuration fields.

Remove Processed opts and everything related to them since they should
not be used anymore.

Fixed test compilation errors caused by the change.
2024-09-05 20:46:40 +02:00
arberkatellari
c7221f0d1e Add reader id within the *vars in ERs 2024-08-12 18:34:50 +02:00
ionutboangiu
d0a435aa6d Migrate to new jetstream API
Upgraded go.mod nats version due to an issue caused by version mismatch
between driver and server (uncertain).

Renamed function from getProcessOptions to getProcessedOptions.

## *NatsER.Serve

- Replaced ChanQueueSubscribe with QueueSubscribe for Core NATS consumer
to handle the message processing directly.

- Since QueueSubscribe is now used regardless of jetstream status, the
message handler has been assigned to a separate variable that can be
reused.

-  The message handler is now dealing with the message processing
directly, therefore the select case listening for the channel which is
feeding NATS messages can be removed together with the channel itself
and the select. Currently, the goroutine within Serve only has to block
until the rdrExit chan is closed.

- Moved the resource check inside the handler right before starting the
message processing goroutine.

## *NatsEE.parseOpts

- Renamed function from parseOpt to parseOpts.

- Handled the error coming from GetNatsOpts function.

## *NatsEE.Connect

- Updated function to return early in case of non-nil nats.Conn value
to reduce nesting.

## *NatsEE.ExportEvent

- Use defer to release resources and RUnlock.

## *NatsEE.Close

- Use defer to Unlock.

- Update function to return early in case of nil nats.Conn value to
reduce nesting.

## ees.GetNatsOpts

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

## ers.GetNatsOpts

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

Removed tab from commented natsJetStreamMaxWaitProcessed option
value in config_defaults.go under ers section.

Added integration test for ERs NATS.

Updated ees/ers implementation to use the jetstream package which
separates the jetstream context from Core NATS.

Removed the jsOpts fields from the NatsEE struct. We are now using the
jetStreamMaxWait option directly through a timeout context.

Added streamName option for NATS reader since it is now required to be
specified when creating a consumer (it is not inferred based on subject
anymore).

Updated nats ers integration tests.

Updated tests to also use the new jetstream package.

Updated tests to start the nats-server using their official driver
instead of using the std go exec package. time.Sleeps are now not
required anymore to wait for the server.

In test configurations for nats readers, made sure that natsStreamName
option is populated. It is now required for consumers to know where to
subscribe.
2023-09-28 20:48:39 +02:00
ionutboangiu
113e2a2bdf Replace interface{} with any 2023-05-31 10:22:28 +02:00
ionutboangiu
924a91c749 Change SSLMode to PgSSLMode 2022-07-03 13:08:52 +02:00
andronache98
ecdf5e54ea Changed headers 2022-03-09 14:14:30 +01:00
andronache98
9ec56f5155 Changed headers 2022-03-09 14:14:30 +01:00
ionutboangiu
8ce6982324 Remove <populated> return parameter from GetProcessOptions 2021-11-01 13:51:02 +02:00
ionutboangiu
316e9d6f32 Change type of ExportEvent opts to *EventExporterOpts and fix compilation errors 2021-11-01 13:51:02 +02:00
ionutboangiu
aa46b1eefd Fix compilation errors after making ers opts into structs 2021-11-01 13:51:02 +02:00
Trial97
c28ac62968 Updated flatstore tests 2021-05-19 15:54:14 +02:00
Trial97
d868c0a5f0 Added partial reader 2021-05-19 15:54:14 +02:00
andronache
ccf014451c Finished renaming opts for Event Exporter 2021-05-11 13:21:48 +02:00
andronache
a7245520a7 Changed sql opts for Event Reader and Event Exporter 2021-05-11 13:21:48 +02:00
Trial97
f6863bad3e Updated flatsore ers 2021-05-06 14:00:37 +02:00
Trial97
3070139cb2 Prepared flatStore 2021-05-06 14:00:37 +02:00
Trial97
7aeeeec50c Updated ers config 2021-05-04 20:45:09 +02:00
Trial97
8aa7009d36 Updated AttributesSv1 APIs 2021-04-13 13:35:03 +02:00
Trial97
c13a5cf790 Finished implementing the DataNode as a replacement 2021-03-26 13:46:25 +01:00
TeoV
2bfb234804 Update integration test in agents 2021-01-15 09:53:55 +01:00
TeoV
906baa051a Build success 2021-01-14 10:30:35 +01:00
Trial97
df702b71aa Finished updating gorm library 2021-01-12 20:01:16 +01:00
Trial97
ca13e8d364 Updated tariffplans gorm statements 2021-01-12 20:01:16 +01:00
Trial97
13f2449376 Started updating to new gorm library 2021-01-12 20:01:16 +01:00
andronache
dbb9216b6e Modified consts in utils 2021-01-07 18:15:35 +01:00
Trial97
06e4fd13c3 Updated integration tests 2020-12-09 17:00:04 +01:00
Trial97
8486697829 Updated integration tests 2020-11-27 08:41:56 +01:00
Trial97
fc9eb41e7e Added AMQPv1 Event Reader 2020-10-20 11:09:16 +02:00
Trial97
228665989e Updated ERs and EEs options 2020-09-04 12:32:09 +02:00
Trial97
f10f570a27 Removed config.NavigableMap 2020-05-18 13:26:06 +02:00
Trial97
13e813f848 Fixed agents unit tests 2020-05-07 11:17:41 +02:00
Trial97
41e35bc571 Added Opts to APIs 2020-04-26 13:08:34 +02:00
TeoV
b85ebe8214 Update EventReader integration tests 2020-02-05 15:57:58 +01:00
TeoV
52f0e0a86c Merge ContentFields,HeaderFields,TrailerFields into one Files 2020-01-29 14:40:57 +01:00
TeoV
26ea469328 Merge content,header,trailer fields into fields 2020-01-29 14:40:57 +01:00
Trial97
546404f4f2 Updated ers unit tests 2019-12-18 17:57:59 +02:00
Trial97
656fb8be00 Updated *sql Event Reader 2019-12-17 11:43:44 +02:00
Trial97
7c8e3810b2 Added *sql Event Reader 2019-12-17 11:43:44 +02:00