2815 Commits

Author SHA1 Message Date
ionutboangiu
16eb745f00 Centralize test flags in utils for shared use across tests 2024-04-08 13:14:02 +02:00
gezimbll
6f4ee914f7 revised meprofile &cpuprofile tests 2024-04-08 13:12:54 +02:00
ionutboangiu
e56191e390 Add ERs APIs to dispatchers 2024-04-04 12:31:18 +03:00
ionutboangiu
a362511d7a Define param struct for ErSv1.RunReader API + refactoring
Rename dir field to sourceDir in file event readers for clarity.
2024-04-04 12:31:18 +03:00
ionutboangiu
0626b050c1 Add ERs apis to apier package
Now the ErSv1 object methods are registered instead when
starting the ERs service.
2024-04-04 12:31:18 +03:00
ionutboangiu
bd4aa99458 Refactor balance's Factor field name with its plural form
Stored balances previous to this commit, that had a non-nil Factors map, will
now have Factors nil due to field name mismatch.
2024-04-01 19:58:04 +03:00
ionutboangiu
ee98dbe0ca Update AgentV1DisconnectSession api signature
Will accept utils.CGREvent instead of utils.AttrDisconnectSession as a
parameter.

SessionSv1.ForceDisconnect will take utils.SessionFilterWithEvent as a
parameter instead of *utils.SessionFilter.

Added possibility to pass DisconnectCause as an Event parameter.

The forceSTerminate that's called when the session timer expires will
have DisconnectCause 'SESSION_TIMEOUT' instead of 'FORCED_DISCONNECT'.

Added Dispatcher methods for AlterSessions.

Event will be merged with EventStart of the session before being sent to
AgentV1DisconnectSession.
2024-03-07 17:20:23 +01:00
arberkatellari
9be32c36da Add CAPs counting to cgr-console status command 2024-03-04 09:50:10 +01:00
ionutboangiu
0aea8ac641 Revert "Move CGREvent from utils to engine"
This reverts commit 02195c47ed.
2024-03-04 09:49:15 +01:00
gezimbll
0c99cf1df5 fixes in integration test 2024-02-28 12:46:17 +01:00
ionutboangiu
02195c47ed Move CGREvent from utils to engine 2024-02-27 17:16:57 +01:00
ionutboangiu
a9079154da Add possibility to pass conn cfg to actions 2024-02-27 16:28:48 +01:00
ionutboangiu
1f7e0b33a2 Update *alter_sessions action
Will now support two extra parameters: address and codec. For *internal
connections, the birpc.Service object will be retrieved from rpcParamsMap
from utils. Supported codecs: <*gob|*json|*http_jsonrpc> (ingored for
*internal address).

Action does not bother with setting defaults anymore, lets the API
handle them.

Improve action comments.

Add unit tests for the action. UnregisterRpcParams' implementation was
required.

Updated *alter_sessions action tariffplan for radius coa integration test
to include address and codec.

Some birpc clients that are set up in integration tests were still registering
a SessionSv1 object. Updated them to register an AgentV1 object instead.
2024-02-27 16:28:48 +01:00
ionutboangiu
f9d1c0f0e1 Refactor ReAuthorize to AlterSessions 2024-02-21 18:43:53 +01:00
ionutboangiu
e50d71808a Optimize TransferBalanceAPI and test *cdrlog functionality 2024-02-20 11:11:39 +01:00
ionutboangiu
87da08f7fd Revise *transfer_balance action and its API
Ensure parameter fields are not abbreviated.

The action will not depend on balance type anymore. It will go through all
balances.

*default balance from source balance can go negative during transfer.
2024-02-19 11:28:52 +01:00
ionutboangiu
0c32f1761c Implement APIerSv1.TransferBalance API 2024-02-19 11:28:52 +01:00
ionutboangiu
267e6109e1 Implement CDRsv1.ProcessEvents API 2024-02-19 11:28:52 +01:00
arberkatellari
e730ef4eb2 Update integration tests 2024-02-15 17:02:41 +01:00
arberkatellari
bbaf75c995 Update readthedocs & add console export_cdrs 2024-02-15 17:02:41 +01:00
ionutboangiu
d8ad760dbb Update both ReAuthorize API signatures
The one from sessions takes an additional event alongside the
SessionFilter, while the one from agents will accept a CGREvent
instead of a simple originID string

The additional event sent to SessionSv1ReAuthorize will be merged
with the EventStart event from the matched session and can be used
when building server initiated requests from the *req map. The
initial packet which was initially inside *req, will be moved to
the *oreq ExtraDP (stands for original request).
2024-02-14 10:15:51 +01:00
arberkatellari
b88045872f Optimise caching_delay 2024-01-09 09:42:48 -05:00
arberkatellari
0cfc4b5658 Continue caching_delay implementation & tests 2024-01-08 11:00:10 -05:00
gezimbll
f258a289f4 Updated statmetrics test && added comments 2023-12-01 18:27:44 +01:00
gezimbll
b99d87c262 Added OneEvent functionality for statmetrics && tests 2023-12-01 18:27:44 +01:00
arberkatellari
953bb6de52 Improve GetAccountCost 2023-11-09 11:19:04 +01:00
arberkatellari
054c584775 Create APIerSV1GetAccountCost 2023-11-09 09:37:34 +01:00
gezimbll
c2dacc42c1 Removed warnings from go staticcheck 2023-11-08 17:10:13 +01:00
gezimbll
5a85c119c8 Replacing IsSliceMember ,CloneStringSlice and SliceStringEqual with slices package equivalent functions 2023-11-01 21:52:50 +01:00
arberkatellari
e3915b2009 Correct test and add test for rerating CDRs using ERs 2023-10-16 13:35:30 +03:00
arberkatellari
47e75fab13 Update radigo, add multiple folder functionality 2023-10-11 11:18:32 +02:00
gezimbll
37e5a54ea9 Fixed config tests & added new stordb tests 2023-10-04 20:58:25 +02:00
ionutboangiu
47af22c724 Update rpcclient library to latest version
`ClientConnector` is no longer defined within `rpcclient` in its latest
version. It has been changed to be obtained from the `cgrates/birpc`
library instead.

Replaced `net/rpc` with `cgrates/birpc` and `net/rpc/jsonrpc` with
`cgrates/birpc/jsonrpc` libraries.

The implementations of `CallBiRPC()` and `Handlers()` were removed,
along with the methods associated with them.

The `rpcclient.BIRPCConector` and the methods prefixed with `BiRPC` were
removed from the `BiRPClient` interface.

The `BiRPClient` interface was renamed to `BIRPCClient`, although not
sure if needed (seems useful just to test if the structure is correct).

`rpcclient.BiRPCConector` has been replaced with `context.ClientConnector`,
which is now passed alongside `context.Context` within the same struct
(`cgrates/birpc/context.Context`). Consequently, functions that were
previously relying on it are now receiving the context instead. The
changes were made in the following functions:

    - `engine/connmanager.go` - `*ConnManager.Call`
    - `engine/connmanager.go` - `*ConnManager.getConn`
    - `engine/connmanager.go` - `*ConnManager.getConnWithConfig`
    - `engine/libengine.go` - `NewRPCPool`
    - `engine/libengine.go` - `NewRPCConnection`
    - `agents/libagents.go` - `processRequest`

Compilation errors related to the `rpcclient.NewRPCClient` function were
resolved by adding the missing `context`, `max_reconnect_interval`, and
`delayFunc` parameters. Additionally, context was added to all calls made
by the client. An effort was made to avoid passing hardcoded values as
much as possible, and extra flags were added where necessary for cgr
binaries.

The `max_reconnect_interval` parameter is now passed from parent
functions, which required adjustments to the function signature.

A new context field was added to all agent objects to ensure access to
it before sending it to the `connmanager's Call`, effectively replacing
`birpcclient`. Although an alternative would have been to create the
new service and add it to the context right before passing it to the
handlers, the chosen approach is definitely more comfortable.

With the addition of a context field for the SIP servers agents, an
additional error needed to be handled, coming from the creation of the
service. Agent constructors within the services package log errors as
they occur and return. Alternate solutions considered were either
shutting down the engine instead of returning, or just logging the
occurrence as a warning, particularly when the `ctx.Client` isn't
required, especially in cases where bidirectional connections are not
needed. For the latter option, it's crucial to return the object with
the error rather than nil or to make the error nil immediately after
logging.

Context has been integrated into all internal Call implementations to
ensure the objects conform to the `birpc.ClientConnector` interface.
These implementations will be removed in the near future as all service
objects are being wrapped in a `birpc.Service` type that satisfies the
`birpc.ClientConnector` interface. Currently, they are being retained
as a reference in case of any unexpected issues that arise.

Ensured that the `birpc.Service` wrapped service objects are passed to
the internal channel getters rather than the objects themselves.

Add context.TODO() to all \*ConnManager.Call function calls. To be
replaced with the context passed to the Method, when available.

For all `*ConnManager.Call` function calls, `context.TODO()` has been
added. This will be replaced with the context passed to the method when
it becomes available.

The value returned by StringGetOpts is now passed directly to the
FirstNonEmpty function, instead of being assigned to a variable
first.

The implementation of the `*AnalyzerService.GetInternalBiRPCCodec`
function has been removed from the services package. Additionally,
the AnalyzerBiRPCConnector type definition and its associated methods
have been removed.

The codec implementation has been revised to include the following
changes:

    - `rpc.ServerCodec` -> `birpc.ServerCodec`;
    - `rpc2.ServerCodec` -> `birpc.BirpcCodec`;
    - `rpc2.Request` -> `birpc.Request`;
    - `rpc2.Response` -> `birpc.Response`;
    - The constructors for json and gob birpc codecs in `cenkalti/rpc`
    have been replaced with ones from the `birpc/jsonrpc` library;
    - The gob codec implementation has been removed in favor of the
    version already implemented in the birpc external library.

The server implementation has been updated with the following changes:

    - A field that represents a simple RPC server has been added to the
    Server struct;
    - Both the simple and bidirectional RPC servers are now initialized
    inside the Server constructor, eliminating the need for nil checks;
    - Usage of `net/rpc` and `cenkalti/rpc2` has been replaced with
    `cgrates/birpc`;
    - Additional `(Bi)RPCUnregisterName` methods have been added;
    - The implementations for (bi)json/gob servers have been somewhat
    simplified.

Before deleting the Call functions and using the `birpc.NewService`
method to register the methods for all cgrates components, update the
Call functions to satisfy the `birpc.ClientConnector` interface. This
way it will be a bit safer. Had to be done for SessionS though.

The `BiRPCCall` method has been removed from coreutils.go. The
`RPCCall` and `APIerRPCCall` methods are also to be removed in the
future.

Ensured that all methods for `SessionSv1` and `SessionS` have the
correct function signature with context. The same adjustments were made
for the session dispatcher methods and for the `SessionSv1Interface`.
Also removed sessionsbirpc.go and smgbirpc.go files.

Implemented the following methods to help with the registration of
methods across all subsystems:

    - `NewServiceWithName`;
    - `NewDispatcherService` for all dispatcher methods;
    - `NewService` for the remaining methods that are already named
    correctly.

Compared to the constructor from the external library, these also make
sure that the naming of the methods is consistent with our constants.

Added context to the Call methods for the mock client connectors (used
in tests).

Removed unused rpc fields from inside the following services:

    - EeS
    - LoaderS
    - ResourceS
    - RouteS
    - StatS
    - ThresholdS
    - SessionS
    - CoreS

Updated the methods implementing the logic for API methods to align
with the latest changes, ensuring consistency and correctness. The
modifications include:

    - Adjusting the function signature to the new format
    (ctx, args, reply).
    - Prefixing names with 'V*' to indicate that they are utilized by
    or registered as APIs.
    - Containing the complete logic within the methods, enabling APIs
    to call them and return their reply directly.

The subsystems affected by these changes are detailed as follows:

    - CoreS: Additional methods were implementing utilizing the
    existing ones. Though modifying them directly was possible, certain
    methods (e.g., StopCPUProfiling()) were used elsewhere and not as
    RPC requests.
    - CDRs: Renamed V1CountCDRs to V1GetCDRsCount.
    - StatS: V1GetQueueFloatMetrics, V1GetQueueStringMetrics,
    V1GetStatQueue accept different arguments compared to API functions
    (opted to register StatSv1 instead).
    - ResourceS: Renamed V1ResourcesForEvent to V1GetResourcesForEvent
    to align with API naming.
    - DispatcherS: Renamed V1GetProfilesForEvent to
    DispatcherSv1GetProfilesForEvent.
    - For the rest, adding context to the function signature was enough.

In the unit tests, wrapping the object within a biprc.Service is now
ensured before passing it to the internal connections map under the
corresponding key.

Some tests that are covering error cases, are also checking the other
return value besides the error. That check has been removed since it
is redundant.

Revised the RPC/BiRPC clients' constructors (for use in tests)

A different approach has been chosen for the handling of ping methods
within subsystems. Instead of defining the same structure in every file,
the ping methods were added inside the Service constructor function.
Though the existing Ping methods were left as they were, they will be
removed in the future.

An additional method has been implemented to register the Ping method
from outside of the engine package.

Implemented Sleep and CapsError methods for SessionS (before they were
exclusively for bidirectional use, I believe).

A specific issue has been fixed within the CapsError SessionSv1 API
implementation, which is designed to overwrite methods that cannot be
allocated due to the threshold limit being reached. Previously, it was
deallocating when writing the response, even when a spot hadn't been
allocated in the first place (due to the cap being hit). The reason
behind this, especially why the test was passing before, still needs
to be looked into, as the problem should have occurred from before.

Implement `*SessionSv1.RegisterInternalBiJSONConn` method in apier.

All agent methods have been registered under the SessionSv1 name. For
the correct method names, the leading "V1" prefix has been trimmed
using the `birpc.NewServiceWithMethodsRename` function.

Revise the RegisterRpcParams function to populate the parameters
while relying on the `*birpc.Service` type instead. This will
automatically also deal with the validation. At the moment,
any error encountered is logged without being returned. Might
be changed in the future.

Inside the cgrRPCAction function, `mapstructure.Decode`'s output parameter
is now guaranteed to always be a pointer.

Updated go.mod and go.sum.

Fixed some typos.
2023-09-01 11:23:54 +02:00
gezimbll
a63b180f2a Integration tests fixes & deleting copr ansible scripts 2023-08-07 11:41:48 +02:00
ionutboangiu
ef4a59c31e Apply gofmt to make workflow pass 2023-07-21 09:00:46 +02:00
gezimbll
2c1f5012fc Updated sentrypeer filter value types 2023-07-14 17:24:03 +02:00
gezimbll
8ed7bdd060 Added it test_for sentrypeer filter 2023-07-14 17:24:03 +02:00
gezimbll
1b1dd13677 Separating opts for each eventexporter type and fixing tests 2023-06-22 10:36:46 +02:00
gezimbll
3fbb546c83 Fixes on integration tests 2023-06-07 10:32:32 +02:00
ionutboangiu
6c16ff320f Replace interface{} with any 2023-05-31 10:23:12 +02:00
ionutboangiu
f2201f3e62 Revise err handling for CSV storage constructor
NewFileCSVStorage() now returns an error besides the storage struct itself, which is
logged and returned instead of calling log.Fatal() which was causing the engine to
crash.

Fixed compilation errors by creating the CSVStorage separately and passing it as an
argument to the TpReader constructor.
2023-05-23 03:41:07 -04:00
ionutboangiu
373df2329c Revise CDR rerating
If the reRate parameter is set to true, also set the refund to true.

The rerate parameter is now no longer hardcoded to true for the RateCDRs API.If
required, the "*rerate" flag must be provided by the caller.

In case CostDetails is not populated, retrieve it from StorDB if possible
and add it to the CGREvent before converting to CDRs.

Now that the refund happens before the debit, revise the expected values for
the testV1CDRsProcessEventWithRefund subtest within the
apier/v1/cdrs_it_test.go file.
2023-04-20 20:06:16 +02:00
ionutboangiu
d262aaac8b Revise CDR rerating
If the reRate parameter is set to true, also set the refund to true.

In case CostDetails is not populated, before attempting to refund try to
retrieve it from StorDB.

Now that the refund happens before the debit, revise the expected values for
the testV1CDRsProcessEventWithRefund subtest within the
apier/v1/cdrs_it_test.go file.
2023-04-19 16:09:41 +02:00
arberkatellari
311d01861d Add Integration test for ResourceSv1GetResourceWithConfig 2023-04-06 11:50:03 +02:00
gezimbll
b221a385e4 changed internal db type to *internal 2023-03-08 19:15:23 +01:00
gezimbll
5c4bf0e538 changing db type constants with leading * 2023-03-08 19:15:23 +01:00
gezimbll
f6ab8b599f IT test for setting balance uuid 2023-02-17 12:18:22 +01:00
ionutboangiu
b27ed9f53e Fix caps error typo 2023-02-16 11:28:36 +01:00
ionutboangiu
7c7c701b2c Revise caps integration tests (error handling) 2023-02-16 11:28:36 +01:00
ionutboangiu
3b0a397cd0 Make caps codec return error through API instead of directly 2023-02-16 11:28:36 +01:00