Commit Graph

5097 Commits

Author SHA1 Message Date
ionutboangiu
d182d6f601 Revise behaviour of balance factor
Now it multiplies instead of dividing.
2024-02-06 10:06:30 +01:00
ionutboangiu
f01ae9fb37 Add compatibility support between *remove_balance and *cdrlog actions 2024-01-30 18:46:48 +01:00
ionutboangiu
8c0bee1a1a Add error constant for INSUFFICIENT_CREDIT_BALANCE_BLOCKER 2024-01-29 09:57:49 +01:00
ionutboangiu
050af6032c Include DestinationIDs and Factor when cloning balances
This fixes an issue where, if data_db is of type *internal, the Factor value is lost
on GetAccount
2024-01-29 09:57:49 +01:00
gezimbll
dfb8011a1a Fix encoding dataprovider as json body for *http filter 2024-01-26 12:11:46 +01:00
ionutboangiu
911cd1b607 Update SMCostRetries backoff increment unit to ms 2024-01-25 18:41:17 +01:00
ionutboangiu
5c96ecf27e Update balance factor to depend on category
Updated unit tests to reflect that also.
2024-01-25 18:41:17 +01:00
ionutboangiu
dca2c4b6a1 Revise unit tests following the blocker update 2024-01-23 14:12:33 +01:00
ionutboangiu
36bdc41e97 Add support for kafka ssl encryption
For both poster and reader.
Added integration test.
2024-01-23 14:12:33 +01:00
DanB
e0d15f1a9e Prevent balance Blocker to partially debit 2024-01-19 16:28:09 +01:00
gezimbll
6f6374abb9 added *http attribute type 2024-01-15 16:17:02 +01:00
gezimbll
584a55ab25 added *http filter type 2024-01-15 16:17:02 +01:00
ionutboangiu
b76d61281c Implement new *transfer_balance action
Added possibility to mock datamanager account functions.

Fixed typo in SubtractValue function name.

Added unit & integration tests.
2024-01-15 16:15:18 +01:00
gezimbll
422d9196b4 Added new filter "*contains" 2023-12-18 17:30:15 +01:00
ionutboangiu
b4ef61d6f2 Update eventcost FieldAsInterface methods
Rating map is now accessible from Accounting.

ExtraCharges struct is accessible from Accounting.

RatingUnit fields that did not represent the id of another event cost struct
are now retrievable.
2023-12-13 20:32:27 +01:00
ionutboangiu
17fbc2d51b Rename mongoScheme opt to mongoConnScheme 2023-12-12 10:10:26 +01:00
ionutboangiu
24cbe49d90 Add support for mongodb+srv conn format
Option can be found in db opts under the name 'mongoSchema'.
2023-12-11 17:15:40 +01:00
ionutboangiu
1154cad0df Rename max_recursion_depth cfg param to fallback_depth 2023-12-07 17:08:22 +01:00
ionutboangiu
ff53446e41 Make recursion_max_depth a configurable option
Also added an integration test checking the recursion depth functionality.
2023-12-07 17:08:22 +01:00
ionutboangiu
7886a35d63 Rename composeURI func to composeMongoURI
To reflect that it's used exclusively for MongoDB (for now at least).

Also added a descriptive comment to the function.
2023-12-07 17:07:42 +01:00
ionutboangiu
a057b34505 Use simple string concatenation to build URI
No need to keep the Parse call as a validation step since
any issue would be caught when establishing a connection.

Renamed buildURL to composeURI.
2023-12-07 17:07:42 +01:00
ionutboangiu
b502920b0b Revise buildURL function
The RawQuery field would not be properly populated in
the old implementation. Has been fixed by calling the
Parse function at the end.
2023-12-07 17:07:42 +01:00
gezimbll
f258a289f4 Updated statmetrics test && added comments 2023-12-01 18:27:44 +01:00
gezimbll
b99d87c262 Added OneEvent functionality for statmetrics && tests 2023-12-01 18:27:44 +01:00
gezimbll
b2aad38e68 Added OneEvent functionality to StatS 2023-12-01 18:27:44 +01:00
gezimbll
5f801c8b67 Revise integration tests 2023-11-30 16:52:43 +01:00
ionutboangiu
a440c18c0c Prolong expiration date for balance in unit test 2023-11-14 18:26:54 +01:00
ionutboangiu
7eeab045a3 Trim fldPath Increments before retrieving one of its fields 2023-11-14 18:23:33 +01:00
arberkatellari
681cd3c58a Add dryrun for CDRsV2ProcessEvent 2023-11-09 09:37:34 +01:00
gezimbll
09e5088262 Removed warnings from go staticcheck 2023-11-08 17:10:13 +01:00
gezimbll
c2dacc42c1 Removed warnings from go staticcheck 2023-11-08 17:10:13 +01:00
gezimbll
5a85c119c8 Replacing IsSliceMember ,CloneStringSlice and SliceStringEqual with slices package equivalent functions 2023-11-01 21:52:50 +01:00
ionutboangiu
6722d6b6a4 Remove unused DryRun field from CallDescriptor
dryRun vars will also not be considered anymore in the balance
blocker condition.

Fixes #4163
2023-11-01 21:37:32 +01:00
ionutboangiu
88f31c189d Optimize CDRs unit test (12s->1s) 2023-10-27 18:51:01 +02:00
ionutboangiu
38578524da Upgrade MongoDB driver to v1.12
- Set (but comment) serverAPI options (currently distinct api and
create.size BSON field are deprecated + possible others that are untested)
- Remove the custom time decoder used for mongo BSON
datetime values. The custom decoder was only converting these values
into UTC and was not any different from the default time.Time
decoder in the MongoDB driver, which also handles BSON string, int64,
and document values.
- Implement 'buildURL' function to connect to mongo (can also be
used for mysql and postgres)
- Update function names, variable names, and comments for clarity
- Replace 'bsonx.Regex' with the Regex primitive for v1.12 compatibility
- Use simple concatenation instead of Sprintf
- Declare 'decimalType' locally, replace global 'decimalType'
- Simplify several functions without altering functionality
- Converting directly from a D to an M is deprecated. We are now decoding
  directly in a M.
- Used errors.As and errors.Is for proper error comparison and assertion
- Revised sloppy reassignments and added missing error checks
2023-10-27 18:51:01 +02:00
ionutboangiu
62b27b569f Make EventCost type vars satisfy the DataStorage interface 2023-10-26 15:55:54 +02:00
gezimbll
969ef4b754 Make go vet pass 2023-10-26 15:52:06 +02:00
gezimbll
9df5f2e02e Added EventCost FieldAsString tests 2023-10-26 15:52:06 +02:00
gezimbll
d560152d55 Improving internal storage coverage tests 2023-10-16 13:34:55 +03:00
gezimbll
405f29ca00 added internal storage tests 2023-10-12 15:06:04 +02:00
gezimbll
e52778794c Coverage tests for internal storage 2023-10-09 21:05:42 +02:00
ionutboangiu
1429445bde Update failing versions it test 2023-10-05 17:23:26 +02:00
gezimbll
37e5a54ea9 Fixed config tests & added new stordb tests 2023-10-04 20:58:25 +02:00
Bas Couwenberg
8894cc19ee Fix spelling errors.
* beacause     -> because
 * disconect    -> disconnect
 * quering      -> querying
 * retrived     -> retrieved
 * syncronous   -> synchronous
 * treshold     -> threshold
 * unsuccesfull -> unsuccessful
2023-09-11 12:24:52 +02:00
ionutboangiu
87c6fcac63 Revise failing tests in engine
For the first test, the connection comparison was removed since we are
expecting an error from the constructor.

The second one has been removed, because there is no way to compare
2 instances of rpcclient.RPCClient from engine. It will probably be
added back when an Equal function will be implemented for RPCClient.
2023-09-01 18:11:54 +02:00
ionutboangiu
47af22c724 Update rpcclient library to latest version
`ClientConnector` is no longer defined within `rpcclient` in its latest
version. It has been changed to be obtained from the `cgrates/birpc`
library instead.

Replaced `net/rpc` with `cgrates/birpc` and `net/rpc/jsonrpc` with
`cgrates/birpc/jsonrpc` libraries.

The implementations of `CallBiRPC()` and `Handlers()` were removed,
along with the methods associated with them.

The `rpcclient.BIRPCConector` and the methods prefixed with `BiRPC` were
removed from the `BiRPClient` interface.

The `BiRPClient` interface was renamed to `BIRPCClient`, although not
sure if needed (seems useful just to test if the structure is correct).

`rpcclient.BiRPCConector` has been replaced with `context.ClientConnector`,
which is now passed alongside `context.Context` within the same struct
(`cgrates/birpc/context.Context`). Consequently, functions that were
previously relying on it are now receiving the context instead. The
changes were made in the following functions:

    - `engine/connmanager.go` - `*ConnManager.Call`
    - `engine/connmanager.go` - `*ConnManager.getConn`
    - `engine/connmanager.go` - `*ConnManager.getConnWithConfig`
    - `engine/libengine.go` - `NewRPCPool`
    - `engine/libengine.go` - `NewRPCConnection`
    - `agents/libagents.go` - `processRequest`

Compilation errors related to the `rpcclient.NewRPCClient` function were
resolved by adding the missing `context`, `max_reconnect_interval`, and
`delayFunc` parameters. Additionally, context was added to all calls made
by the client. An effort was made to avoid passing hardcoded values as
much as possible, and extra flags were added where necessary for cgr
binaries.

The `max_reconnect_interval` parameter is now passed from parent
functions, which required adjustments to the function signature.

A new context field was added to all agent objects to ensure access to
it before sending it to the `connmanager's Call`, effectively replacing
`birpcclient`. Although an alternative would have been to create the
new service and add it to the context right before passing it to the
handlers, the chosen approach is definitely more comfortable.

With the addition of a context field for the SIP servers agents, an
additional error needed to be handled, coming from the creation of the
service. Agent constructors within the services package log errors as
they occur and return. Alternate solutions considered were either
shutting down the engine instead of returning, or just logging the
occurrence as a warning, particularly when the `ctx.Client` isn't
required, especially in cases where bidirectional connections are not
needed. For the latter option, it's crucial to return the object with
the error rather than nil or to make the error nil immediately after
logging.

Context has been integrated into all internal Call implementations to
ensure the objects conform to the `birpc.ClientConnector` interface.
These implementations will be removed in the near future as all service
objects are being wrapped in a `birpc.Service` type that satisfies the
`birpc.ClientConnector` interface. Currently, they are being retained
as a reference in case of any unexpected issues that arise.

Ensured that the `birpc.Service` wrapped service objects are passed to
the internal channel getters rather than the objects themselves.

Add context.TODO() to all \*ConnManager.Call function calls. To be
replaced with the context passed to the Method, when available.

For all `*ConnManager.Call` function calls, `context.TODO()` has been
added. This will be replaced with the context passed to the method when
it becomes available.

The value returned by StringGetOpts is now passed directly to the
FirstNonEmpty function, instead of being assigned to a variable
first.

The implementation of the `*AnalyzerService.GetInternalBiRPCCodec`
function has been removed from the services package. Additionally,
the AnalyzerBiRPCConnector type definition and its associated methods
have been removed.

The codec implementation has been revised to include the following
changes:

    - `rpc.ServerCodec` -> `birpc.ServerCodec`;
    - `rpc2.ServerCodec` -> `birpc.BirpcCodec`;
    - `rpc2.Request` -> `birpc.Request`;
    - `rpc2.Response` -> `birpc.Response`;
    - The constructors for json and gob birpc codecs in `cenkalti/rpc`
    have been replaced with ones from the `birpc/jsonrpc` library;
    - The gob codec implementation has been removed in favor of the
    version already implemented in the birpc external library.

The server implementation has been updated with the following changes:

    - A field that represents a simple RPC server has been added to the
    Server struct;
    - Both the simple and bidirectional RPC servers are now initialized
    inside the Server constructor, eliminating the need for nil checks;
    - Usage of `net/rpc` and `cenkalti/rpc2` has been replaced with
    `cgrates/birpc`;
    - Additional `(Bi)RPCUnregisterName` methods have been added;
    - The implementations for (bi)json/gob servers have been somewhat
    simplified.

Before deleting the Call functions and using the `birpc.NewService`
method to register the methods for all cgrates components, update the
Call functions to satisfy the `birpc.ClientConnector` interface. This
way it will be a bit safer. Had to be done for SessionS though.

The `BiRPCCall` method has been removed from coreutils.go. The
`RPCCall` and `APIerRPCCall` methods are also to be removed in the
future.

Ensured that all methods for `SessionSv1` and `SessionS` have the
correct function signature with context. The same adjustments were made
for the session dispatcher methods and for the `SessionSv1Interface`.
Also removed sessionsbirpc.go and smgbirpc.go files.

Implemented the following methods to help with the registration of
methods across all subsystems:

    - `NewServiceWithName`;
    - `NewDispatcherService` for all dispatcher methods;
    - `NewService` for the remaining methods that are already named
    correctly.

Compared to the constructor from the external library, these also make
sure that the naming of the methods is consistent with our constants.

Added context to the Call methods for the mock client connectors (used
in tests).

Removed unused rpc fields from inside the following services:

    - EeS
    - LoaderS
    - ResourceS
    - RouteS
    - StatS
    - ThresholdS
    - SessionS
    - CoreS

Updated the methods implementing the logic for API methods to align
with the latest changes, ensuring consistency and correctness. The
modifications include:

    - Adjusting the function signature to the new format
    (ctx, args, reply).
    - Prefixing names with 'V*' to indicate that they are utilized by
    or registered as APIs.
    - Containing the complete logic within the methods, enabling APIs
    to call them and return their reply directly.

The subsystems affected by these changes are detailed as follows:

    - CoreS: Additional methods were implementing utilizing the
    existing ones. Though modifying them directly was possible, certain
    methods (e.g., StopCPUProfiling()) were used elsewhere and not as
    RPC requests.
    - CDRs: Renamed V1CountCDRs to V1GetCDRsCount.
    - StatS: V1GetQueueFloatMetrics, V1GetQueueStringMetrics,
    V1GetStatQueue accept different arguments compared to API functions
    (opted to register StatSv1 instead).
    - ResourceS: Renamed V1ResourcesForEvent to V1GetResourcesForEvent
    to align with API naming.
    - DispatcherS: Renamed V1GetProfilesForEvent to
    DispatcherSv1GetProfilesForEvent.
    - For the rest, adding context to the function signature was enough.

In the unit tests, wrapping the object within a biprc.Service is now
ensured before passing it to the internal connections map under the
corresponding key.

Some tests that are covering error cases, are also checking the other
return value besides the error. That check has been removed since it
is redundant.

Revised the RPC/BiRPC clients' constructors (for use in tests)

A different approach has been chosen for the handling of ping methods
within subsystems. Instead of defining the same structure in every file,
the ping methods were added inside the Service constructor function.
Though the existing Ping methods were left as they were, they will be
removed in the future.

An additional method has been implemented to register the Ping method
from outside of the engine package.

Implemented Sleep and CapsError methods for SessionS (before they were
exclusively for bidirectional use, I believe).

A specific issue has been fixed within the CapsError SessionSv1 API
implementation, which is designed to overwrite methods that cannot be
allocated due to the threshold limit being reached. Previously, it was
deallocating when writing the response, even when a spot hadn't been
allocated in the first place (due to the cap being hit). The reason
behind this, especially why the test was passing before, still needs
to be looked into, as the problem should have occurred from before.

Implement `*SessionSv1.RegisterInternalBiJSONConn` method in apier.

All agent methods have been registered under the SessionSv1 name. For
the correct method names, the leading "V1" prefix has been trimmed
using the `birpc.NewServiceWithMethodsRename` function.

Revise the RegisterRpcParams function to populate the parameters
while relying on the `*birpc.Service` type instead. This will
automatically also deal with the validation. At the moment,
any error encountered is logged without being returned. Might
be changed in the future.

Inside the cgrRPCAction function, `mapstructure.Decode`'s output parameter
is now guaranteed to always be a pointer.

Updated go.mod and go.sum.

Fixed some typos.
2023-09-01 11:23:54 +02:00
NikolasPetriti
9b0116516e Add tests for msgpack encoding/decoding for all basic go types that we are using in master 2023-08-28 12:22:33 +02:00
NikolasPetriti
cf5e31ce3a Revise struct tags for tp models 2023-08-28 12:22:33 +02:00
gezimbll
cf720550cb Improving coverage tests 2023-08-04 16:01:19 +02:00
gezimbll
763310e46c Improving and fixing tests 2023-08-02 16:48:48 +02:00