48 Commits

Author SHA1 Message Date
ionutboangiu
8443af3a89 Relicense from GPLv3 to AGPLv3 2025-10-29 19:42:40 +01:00
arberkatellari
73667f343f Improve filters validation 2025-07-02 21:46:45 +02:00
arberkatellari
b8cc20bb7c Add sqlBatchSize to ERS SQL OPTS and adjust delete functionality 2025-01-14 17:04:18 +01:00
arberkatellari
d35b14b6b9 Add ability to ERS to update or move ERS SQL events to a new table & add *export flag for ERS readers 2024-12-20 13:26:06 +01:00
arberkatellari
6c752ac10f Improvements to ERS SQL filters 2024-12-03 18:36:22 +01:00
arberkatellari
041b14fa03 Improvements to ERS SQL filters 2024-12-03 18:36:22 +01:00
arberkatellari
321910d181 Add WHERE statement availability and non-delete option to ERS SQL reader 2024-12-03 18:36:22 +01:00
gezimbll
e48eb5df13 added StartDelay for ers processing 2024-11-15 20:58:10 +01:00
arberkatellari
0bab7d5557 Add reader id within the *vars in ERs 2024-08-09 09:14:24 +02:00
gezimbll
5f941b1f9d changed conReqs to not populate buffer at start 2024-04-24 17:09:54 +02:00
ionutboangiu
0aea8ac641 Revert "Move CGREvent from utils to engine"
This reverts commit 02195c47ed.
2024-03-04 09:49:15 +01:00
ionutboangiu
02195c47ed Move CGREvent from utils to engine 2024-02-27 17:16:57 +01:00
ionutboangiu
0c91933065 Revise ERs event exporting
Add ees_success_ids and ees_failed_ids fields in reader config. The
former will be used to set EeIDs when the event processing returns
no error, while the latter will be used otherwise.

Add config sanity checks for the added options.

Remove Processed opts and everything related to them since they should
not be used anymore.

Fixed test compilation errors caused by the change.
2023-11-29 16:21:51 +01:00
ionutboangiu
a80fe9b96e Remove redundant prefixes/suffixes from rOpts field names 2023-09-28 20:48:07 +02:00
ionutboangiu
b1d68b6cfe Upgrade NATS driver version and implementation
Upgraded NATS server version in ansible role.

upgraded go.mod nats version due to an issue caused by version mismatch
between driver and server (uncertain).

Increased amount of sleep in tests after starting a NATS server with
jetstream enabled from 50ms to 100ms.

## *NatsER.Serve

- Replaced ChanQueueSubscribe with QueueSubscribe for Core NATS consumer
to handle the message processing directly.

- Since QueueSubscribe is now used regardless of jetstream status, the
message handler has been assigned to a separate variable that can be
reused.

-  The message handler is now dealing with the message processing
directly, therefore the select case listening for the channel which is
feeding NATS messages can be removed together with the channel itself
and the select. Currently, the goroutine within Serve only has to block
until the rdrExit chan is closed.

- Moved the resource check inside the handler right before starting the
message processing goroutine.

## ers.getProcessedOptions

- Renamed function from getProcessOptions to getProcessedOptions.

- Initially, the EventExporterOpts struct was always initialized to be
non-nil which meant exporting processed messages by the reader was
always enabled by force. Function has been updated to initialize it
only once when the first Processed option that was set is found.

- Created init function for all types of opts structs to avoid
repetition.

## *NatsEE.parseOpts

- Renamed function from parseOpt to parseOpts.

- Updated function to return early in case of nil opts struct to reduce
nesting.

- The nested jetstream status and maxwait conditions nil verification
have been merged into one condition.

- Handled the error coming from GetNatsOpts function.

- NATS Subject assignment has been removed. It was redundant, since it
had already been set from before this function was called.

## *NatsEE.Connect

- Updated function to return early in case of non-nil nats.Conn value
to reduce nesting.

## *NatsEE.ExportEvent

- Use defer to release resources and RUnlock.

## *NatsEE.Close

- Use defer to Unlock.

- Update function to return early in case of nil nats.Conn value to
reduce nesting.

## ees.GetNatsOpts

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

## ers.GetNatsOpts

- Passed the NATSROpts struct directly to the function.

- Chose switch over if else when parsing client certificate and keys
opts.

- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.

Removed tab from commented natsJetStreamMaxWaitProcessed option
value in config_defaults.go under ers section.

Removed redundant NATS prefix from config.NATSROpts type field names.

Added function in ers/lib_test.go to create a config file in /tmp
based on a config string.

Added integration test for ERs NATS.
2023-09-26 21:29:52 +02:00
gezimbll
01d3c5d0fa Separating opts in ERS && fixing tests 2023-06-27 11:08:34 +02:00
gezimbll
1b1dd13677 Separating opts for each eventexporter type and fixing tests 2023-06-22 10:36:46 +02:00
ionutboangiu
6c16ff320f Replace interface{} with any 2023-05-31 10:23:12 +02:00
ionutboangiu
62e878574e Solve the remaining compilation errors and replace SSLMode with PgSSLMode 2022-07-12 17:59:09 +02:00
ionutboangiu
38447e9229 Make ers opts as struct and fix compilation errors 2021-11-29 20:57:39 +01:00
Trial97
73de017591 Updated flatstore tests 2021-05-17 18:00:31 +02:00
Trial97
cae5b4ced5 Added partial reader 2021-05-17 18:00:31 +02:00
Trial97
d53b6175d5 Updated flatsore ers 2021-05-17 18:00:31 +02:00
Trial97
8787d463eb Prepared flatStore 2021-05-17 18:00:31 +02:00
andronache
99a50502f7 Finished renaming opts for Event Exporter 2021-05-10 16:46:42 +02:00
andronache
d20f8dac7b Changed sql opts for Event Reader and Event Exporter 2021-05-10 16:46:42 +02:00
andronache
bc4979b74a Renamed opts for DataDB and StorDB 2021-05-06 13:57:30 +02:00
nickolasdaniel
3fd5237e14 Coverage tests for ers 2021-04-12 17:53:41 +02:00
Trial97
c13a5cf790 Finished implementing the DataNode as a replacement 2021-03-26 13:46:25 +01:00
TeoV
2bfb234804 Update integration test in agents 2021-01-15 09:53:55 +01:00
TeoV
906baa051a Build success 2021-01-14 10:30:35 +01:00
Trial97
df702b71aa Finished updating gorm library 2021-01-12 20:01:16 +01:00
Trial97
ca13e8d364 Updated tariffplans gorm statements 2021-01-12 20:01:16 +01:00
Trial97
13f2449376 Started updating to new gorm library 2021-01-12 20:01:16 +01:00
andronache
dbb9216b6e Modified consts in utils 2021-01-07 18:15:35 +01:00
Trial97
06e4fd13c3 Updated integration tests 2020-12-09 17:00:04 +01:00
Trial97
8486697829 Updated integration tests 2020-11-27 08:41:56 +01:00
Trial97
fc9eb41e7e Added AMQPv1 Event Reader 2020-10-20 11:09:16 +02:00
Trial97
228665989e Updated ERs and EEs options 2020-09-04 12:32:09 +02:00
Trial97
f10f570a27 Removed config.NavigableMap 2020-05-18 13:26:06 +02:00
Trial97
13e813f848 Fixed agents unit tests 2020-05-07 11:17:41 +02:00
Trial97
41e35bc571 Added Opts to APIs 2020-04-26 13:08:34 +02:00
TeoV
b85ebe8214 Update EventReader integration tests 2020-02-05 15:57:58 +01:00
TeoV
52f0e0a86c Merge ContentFields,HeaderFields,TrailerFields into one Files 2020-01-29 14:40:57 +01:00
TeoV
26ea469328 Merge content,header,trailer fields into fields 2020-01-29 14:40:57 +01:00
Trial97
546404f4f2 Updated ers unit tests 2019-12-18 17:57:59 +02:00
Trial97
656fb8be00 Updated *sql Event Reader 2019-12-17 11:43:44 +02:00
Trial97
7c8e3810b2 Added *sql Event Reader 2019-12-17 11:43:44 +02:00