Commit Graph

13748 Commits

Author SHA1 Message Date
arberkatellari
2e4d8bdcf4 Remove unnecessary LazySanityCheck 2024-01-06 17:12:02 +01:00
arberkatellari
ca01cbcd6b Replacing old log_level from configs with new logger->level 2024-01-06 17:12:02 +01:00
gezimbll
b3956da736 Added new filter *contains 2023-12-19 16:03:35 +01:00
gezimbll
c51122fd20 Moving apis into each subsystem packages 2023-12-13 20:33:09 +01:00
ionutboangiu
5228a641e3 Add integration test for cdr apis 2023-12-13 12:47:02 +01:00
ionutboangiu
8ee919319b Register the CDRServer from cdrs instead of the one from services
Also renamed services.CDRServer to CDRService to prevent such mistakes from
happening in the future.
2023-12-13 12:47:02 +01:00
ionutboangiu
00a94d092f Use birpc service constructor and ensure error is handled 2023-12-12 10:09:18 +01:00
ionutboangiu
555a143b79 Take out cdrID from SetCDR parameters 2023-12-12 10:09:18 +01:00
ionutboangiu
d29f84169c Move cdrs to its own package
Moved engine/cdr.go to utils/cdr.go.
Moved engine/cdrs.go and engine/cdrs_test.go to cdrs package.
Moved api functions from cdrs/cdrs.go to cdrs/apis.go. Same for the
tests.
Deleted CDRsV1 type and its methods.
2023-12-12 10:09:18 +01:00
ionutboangiu
1cec3751e4 Update SetCDR signature to include the cdrID
Updated mongo indexing for cdrs to take into consideration only the
opts.*cdrID field.

Fixed cdrs_it_test.go test.

Before storing CDRs in CDR processEvents, we are now temporarily generating
the *cdrID in APIOpts (until we decide where this should happen).
2023-12-12 10:09:18 +01:00
ionutboangiu
9e56246ba1 Update storage sql scripts for setting up cdr tables 2023-12-12 10:09:18 +01:00
ionutboangiu
5f4da20881 Add *refund dynamic opt for cdrs section 2023-12-12 10:09:18 +01:00
ionutboangiu
9273761ed2 Use the proper cfg section to set log level 2023-12-12 10:09:18 +01:00
ionutboangiu
2ccd6ca74d Add context.Context in SetCDR func signature 2023-12-12 10:09:18 +01:00
ionutboangiu
ab26ba2d02 Implement RemoveCDRs for MongoStorage (untested and not optimized) 2023-12-12 10:09:18 +01:00
ionutboangiu
fb6b38850c Update mongo SetCDR method to take into account allowUpdate param 2023-12-12 10:09:18 +01:00
ionutboangiu
560f42bf8d Add store block back inside *CDRServer.processEvents
Added *store and *rerate dynamic opts for cdrs.
2023-12-12 10:09:18 +01:00
gezimbll
3fe5f70dc9 Adding OneEvent functionality for stats 2023-12-07 17:44:33 +01:00
ionutboangiu
f0852b645b Rename composeURI func to composeMongoURI
To reflect that it's used exclusively for MongoDB (for now at least).

Also added a descriptive comment to the function.
2023-12-07 16:56:08 +01:00
ionutboangiu
8909786a4b Revise buildURL function
Renamed to composeURI.

Use simple string concatenation to build URI because there is
no need to keep the Parse call as a validation step. Any issue
would be caught when establishing a connection.
2023-12-07 16:56:08 +01:00
gezimbll
cc07c5366a Revise integration tests 2023-11-29 16:25:18 +01:00
gezimbll
ac32b7ce60 Added function to cast custom map types 2023-11-29 16:25:18 +01:00
gezimbll
3403a91a2b Revise integration tests 2023-11-29 16:25:18 +01:00
ionutboangiu
031cf1fe43 Add rpc caching for ProcessStoredEvents API 2023-11-29 16:22:58 +01:00
ionutboangiu
68152ecc62 Consider admins' enabled value for stordb ShouldRun() 2023-11-17 11:22:34 +01:00
ionutboangiu
6826f13cc5 Update CDR processEvent to process multiple CGREvents 2023-11-16 15:38:18 +01:00
ionutboangiu
a817f30c2a Rename PrepareFilters to GetFilters 2023-11-16 15:38:18 +01:00
ionutboangiu
f5d69cb41b Implement ProcessStoredEvents API 2023-11-16 15:38:18 +01:00
ionutboangiu
e218ca06a8 Add consts for AdminSv1 CDRs api names 2023-11-15 17:25:21 +01:00
ionutboangiu
1ef2afaa97 Update/remove dependencies
Remove cenkalti/rpc2 dependency and update client constructor.
Commented tests that only start and stop engine.

Fixed compilation errors caused by Azure/go-amqp update.

Remove cgrates/ugocodec dependency since we have everything
we need in ugoriji/go/codec and add tests for it.
2023-11-15 14:03:21 +01:00
ionutboangiu
26cdb571b8 Upgrade MongoDB driver to 1.13
- Set (but comment) serverAPI options (currently distinct api and
create.size BSON field are deprecated + possible others that are untested)
- Remove the custom time decoder used for mongo BSON
datetime values. The custom decoder was only converting these values
into UTC and was not any different from the default time.Time
decoder in the MongoDB driver, which also handles BSON string, int64,
and document values.
- Implement 'buildURL' function to connect to mongo (can also be
used for mysql and postgres)
- Update function names, variable names, and comments for clarity
- Replace 'bsonx.Regex' with the Regex primitive (deprecated since 1.12).
- Use simple concatenation instead of Sprintf
- Declare 'decimalType' locally, replace global 'decimalType'
- Simplify several functions without altering functionality
- Converting directly from a D to an M is deprecated. We are now decoding
  directly in a M.
- Used errors.As and errors.Is for proper error comparison and assertion
- Revised sloppy reassignments and added missing error checks
2023-11-14 18:24:21 +01:00
ionutboangiu
e5ab21def5 Implement AdminSv1 Get/RemoveCDRs APIs for CDRs
Addition of storDB to AdminSv1 was required.
2023-11-09 20:50:36 +01:00
ionutboangiu
d5039b3ce8 Add CDRs CRUD queries back and remove TP functions 2023-11-09 20:50:36 +01:00
ionutboangiu
6b831d3ba3 Make Paginate function generic
Also move the paginator implementation and its tests to their
own files.
2023-11-09 20:50:36 +01:00
ionutboangiu
d7679e2e9a Add negative filters to filter type string sets 2023-11-09 20:50:36 +01:00
gezimbll
32ed816de7 Replacing IsSliceMember ,CloneStringSlice and SliceStringEqual with slices package functions 2023-11-01 21:38:27 +01:00
ionutboangiu
65310e7437 Move benchmark relying on redis to it_test file 2023-10-27 18:52:05 +02:00
ionutboangiu
46e39770a1 Replace deprecated Publish method with PublishWithContext 2023-10-27 18:52:05 +02:00
ionutboangiu
7fd7167aa5 Update expected value in kafka it test 2023-10-26 16:26:27 +02:00
ionutboangiu
c47a910386 Update README links to docs 2023-10-26 16:26:27 +02:00
ionutboangiu
b56de12ee1 Bump mongodb version in ansible role 2023-10-26 16:26:27 +02:00
gezimbll
c6571492fc Make go vet pass 2023-10-26 15:59:34 +02:00
gezimbll
7c80bd1002 Remove unreachable err cases from HandleJSONErr
json.UnmarshalError happens only when we pass a non-nil pointer, which
is not the case for us. Usage of go vet helps us make sure it also
won't be happening in the future
2023-10-26 15:59:34 +02:00
arberkatellari
0ca5d3ad20 Change default storedb password 2023-10-25 16:37:19 +02:00
ionutboangiu
6bcc374d45 Update EndOfMonth coreutils test 2023-10-19 17:19:30 +02:00
ionutboangiu
ccd4a92432 Update github workflows 2023-10-19 17:19:30 +02:00
ionutboangiu
4c34d321de Add sanity check to prevent xml reader panic
HierarchyPath parser now returns nil when
the path is empty (instead of a string slice with one
EmptyString element).

If isAbsolute is set to true, when calling the AsString
method on a nil HierarchyPath, only the separator will
be returned. Alternatively, if isAbsolute is false, it
will just return . to signal that the path currently
being processed is the one required.This avoids a nil
expr error coming from the xmlquery library.

Use the Query and QueryAll functions from the xmlquery
package to be able to handle the errors ourselves and
avoid panics.

Remove config default value for xmlRootPath. The field
will remain commented in config_defaults for reference.

Add tests for HierarchyPath.AsString function.

Add comments for XmlProvider and xmlRootPath opt.

Update XmlProvider String() func based on the comment
which specifies that it should display the already
parsed values out of cache.
2023-10-19 17:19:30 +02:00
ionutboangiu
dffc617e17 Update kafka handler to run with sudo 2023-10-19 17:19:30 +02:00
ionutboangiu
2869728de0 Handle kafka topic creation/deletion within test 2023-10-17 17:39:14 +02:00
ionutboangiu
5f079fbca8 Update kafka ansible role
- use kraft instead of zookeeper
- add handlers in case of cfg changes
- create a separate user for the kafka service
- bump kafka version
- make the role more configurable
2023-10-17 17:39:14 +02:00