Commit Graph

2642 Commits

Author SHA1 Message Date
arberkatellari
0bab7d5557 Add reader id within the *vars in ERs 2024-08-09 09:14:24 +02:00
ionutboangiu
d0183597f1 Add postgres stor_db SSL opts 2024-08-09 09:13:10 +02:00
ionutboangiu
78e020e2be Implement HTTP endpoint for prometheus
configurable via 'prometheus_url' option in the HTTP section
2024-08-05 13:28:33 +02:00
gezimblliku
22b96ca1e3 added tests for filter statmetric 2024-08-04 21:10:50 +02:00
ionutboangiu
63129feb4c prevent memprof file overwrites for small intervals
Previously, the timestamp was accurate only down to seconds. For smaller intervals,
this would truncate the previous file(s). Now, for small intervals, the number of
microseconds is appended to the file name.

Added the possibility to disable timestamps in the memory profile file names and use
increments of 1 instead.

Updated the memory profiling integration tests.
2024-08-02 09:35:36 +02:00
ionutboangiu
1c490a9020 Revise memory profiling implementation
- merge StopChanMemProf with StopMemoryProfiling
- remove fileMEM and stopMemProf from struct and constructors
- add separate mutex for memory profiling, ensure thread safety
- handle all significant errors
- log error if StopMemoryProfiling fails during CoreS Shutdown
- ignore errors if profiling inactive in Shutdown and deferred Stop
- move validations inside V1 functions
- return error if StartMemoryProfiling already started
- return error if StopMemoryProfiling already stopped or never started
- close profiling loop on error, not the cgr-engine
- StopMemoryProfiling closes channel and profiling loop writes final profile
- rename Path to DirPath for mandatory field error
- rename memprof_nrfiles flag to memprof_maxfiles
- increase default memprof_interval
- consider MaxFiles <= 0 as unlimited
- move memory profiling logic after starting services
- use CoreService Start/StopMemoryProfiling in main
- remove final memory profile block (created by deferred Stop)
- convert MemProfiling to method on CoreService and rename to profileMemory
- use Ticker for recurrent actions instead of Timer
- compute mem_final.prof full path in StartMemoryProfiling
- suffix profile files with current time instead of numbers
- update dispatcher methods after changes
- move MemoryPrf from utils to cores, rename to MemoryProfilingParams
- add logs for starting/stopping profiling
2024-08-02 09:35:36 +02:00
ionutboangiu
6f4d2144a6 Add test & constant for set_versions flag
Also added set_versions help entry to cgr-engine doc.
2024-08-02 09:35:36 +02:00
ionutboangiu
958aa267cf Refactor HTTP profiling and flag handling
- hardcode the base path for profiling endpoints to '/debug/pprof/'
- change profiling flag 'httprof_path' to boolean 'http_pprof'
  (because of the above)
- remove redundant profiling endpoint registrations (handled by pprof.Index)
- move profiling registration log after actual registration
- make profiling registration log more descriptive
- use utils.Logger instead of log.Print for the log mentioned above
- refactor flags test into a table test (adding also verification for default
  flag values)
- change 'scheduledShutdown' flag type from string to time.Duration (to avoid
  unnecessary time parsing)
- revise flags usage descriptions
- rename flag 'singlecpu' to 'singleCPU'
- switch to 'ExitOnError' for flag parsing to simplify error handling and
  automatically handle 'ErrHelp' by exiting with status 0 when help is
  requested and status 2 for other parsing errors. Before the following error
  log would have been received:
  '<InitS> error received: <flag: help requested>, exiting!'
- update cgr-engine documentation
2024-08-02 09:35:36 +02:00
ionutboangiu
4cba4a699d Implement SharedActionsData for action groups
- holds common relevant data between actions.
- currently supports only *cdrlog, making the process more reliable
and less repetitive. It also provides access to more information about
actions when creating CDRs.
- easier to scale.
- removes the need for cloning accounts before executing a group of
actions.
- added Actions method to check if specific action types exist inside.
2024-07-21 19:04:53 +02:00
gezimbll
85bd7a50e7 updated sags service name to ranking 2024-07-14 13:39:20 +02:00
gezimbll
4d4392d196 revised comments && renamed sars to trends 2024-07-14 13:39:20 +02:00
gezimbll
6c2fbbec28 added sars model,tp,api 2024-07-14 13:39:20 +02:00
ionutboangiu
eb3bd5cc1d Add redisPoolPipelineLimit configuration field for datadb
Sets the maximum number of commands that can be pipelined before flushing.
0 means no limit and pipelines will only be limited by the time window.
2024-07-05 11:21:16 +02:00
ionutboangiu
ed3fa0d294 Add redisPoolPipelineWindow configuration field for datadb
Sets the duration after which internal pipelines are flushed.
0 disabled implicit pipelining.
2024-07-02 19:58:03 +02:00
ionutboangiu
4d3868aad0 Add support for Git 2.45+ iso-strict date format
Git 2.45+ introduced a backward incompatible change in the iso-strict
date format, showing time in the Zulu timezone with Z suffix instead
of +00:00. This commit adds parsing for the new date format before
falling back to the old format.

Ignore errors for old git versions where %cI is not defined.

Revise GetCGRVersion error messages.
Revise GetCGRVersion unit test.
2024-06-30 17:53:38 +02:00
gezimbll
777d4f0a28 added caching for sagprofile 2024-06-30 17:45:06 +02:00
gezimbll
0012477d52 added stat aggregator profile,models,tps and apis 2024-06-30 17:45:06 +02:00
ionutboangiu
11b96de00a Add kafkaBatchSize configuration field
Avoids the default 1 second delay when the batch doesn't
reach 100 messages within that time.

Useful when the Kafka exporter is not cached, as it would
otherwise encounter that delay. Setting BatchSize to 1
prevents this.
2024-06-30 16:28:52 +02:00
ionutboangiu
49f6c5982e Add reference value functionality to *transfer_balance action
The *transfer_balance action can now use a reference value to ensure
the destination balance reaches a specified amount. If the destination
balance exceeds the reference value, the excess is transferred back
to the source balance. If the destination balance is below the
reference value, the required amount is transferred from the source
balance to the destination balance to reach the specified reference
value. An error is returned if the transfer cannot achieve the
specified reference value.

Used by specifying DestinationReferenceValue inside ExtraParameters.

Other *transfer_balance changes:
- used json tags when unmarshaling ExtraParameters in order to be
able to shorten the names of the fields
- lock the destination account only if it's different from the
source account. It is still passed to the Guard function but
without a lock key and with 0 timeout.
- if the transfer happens within the same account, update the
account and execute its ActionTriggers only once.
- moved transfer units validation after retrieving/creating the
destination balance

*cdrlog action has been updated to create cdrs for reference
*transfer_balance actions, although improvements are needed and
the functionality is not completely tested.

APIerSv1.TransferBalance has been updated to take into account the
ReferenceValue parameter.

Added new *transfer_balance action unit tests to account for the
new changes.

Added integration tests (incomplete for now, but functionality
has been tested manually).
2024-06-13 13:56:40 +02:00
arberkatellari
a0f94c6804 Add/Update unit and integration tests adjusting for sessions backup 2024-06-12 18:21:50 +02:00
arberkatellari
f356695f6f Add active sessions backup functionalitiy 2024-06-12 18:21:50 +02:00
gezimbll
1c1d211c7c added config,service for sags 2024-06-12 14:53:30 +02:00
armirveliaj
f707236a37 Add unit test on utils 2024-06-11 18:49:39 +02:00
ionutboangiu
d31b4d0984 Add engine flag to print the config object in JSON format
Removed the redundant check_config engine flag. Config sanity was
checked by the NewCGRConfigFromPath function.
2024-06-11 18:34:02 +02:00
gezimbll
8103cd4c9f renamed sarS api&&register the sarS service to servmanager 2024-06-11 10:21:51 +02:00
gezimbll
86df598920 Implemented service,config and api for Sars 2024-06-11 10:21:51 +02:00
gezimbll
8bf51a6112 added URLEncoder data converter 2024-06-07 16:40:54 +02:00
gezimbll
2293f7f66d added urldecode data converter 2024-06-07 16:40:54 +02:00
armirveliaj
b4f889247f Add unit tests on utils 2024-06-06 13:17:59 +02:00
armirveliaj
1ef9ea768e Add unit test for orderedmap 2024-05-16 16:28:21 +02:00
gezimbll
64692891bb added V1GetActiveSessionIDs method for JanusAgent && revised tests 2024-05-16 16:24:50 +02:00
gezimbll
cf720b0618 added postgres configuration option to choose the schema(#4318) 2024-04-16 12:23:38 +02:00
gezimbll
a3e3cde958 added janusconn methods for loading from json 2024-04-09 20:22:31 +02:00
DanB
8db91ca479 Basic service infrastructure for JanusAgent 2024-04-08 20:09:06 +02:00
ionutboangiu
a5d29d5d7b Stop sending BalanceUpdate events to thresholds/stats 2024-04-08 17:34:13 +02:00
ionutboangiu
bbf441f83c Add flaky build constrait to unstable tests 2024-04-08 13:14:02 +02:00
ionutboangiu
16eb745f00 Centralize test flags in utils for shared use across tests 2024-04-08 13:14:02 +02:00
ionutboangiu
e0ae593949 Set up internal connection to ERs 2024-04-04 12:31:18 +03:00
ionutboangiu
b9a39e233f Implement ErSv1.ProcessDir api
cores.Server, analyzer object and internal conn channel specific to ERs
are now part of the ERService struct and are passed to its constructor.
2024-04-04 12:31:18 +03:00
ionutboangiu
b9b07dc561 Remove redundant dir parameter from processFile methods
Applies to both file readers and loader (for loader, the blank statement
was used anyway).

It's redundant because for file readers, the rdr.dir value was always
passed as the parameter when it was already part of the method's object.

Parameter had to also be removed from the WatchDir function and the
functions it depends on.
2024-04-04 12:31:18 +03:00
ionutboangiu
6db11e87bc Rename Factor to BalanceFactor inside Accounting 2024-04-01 20:48:57 +03:00
ionutboangiu
50d0103c7b Move BalanceFactor value from Increments to Accounting
Retrieve the Accounting BalanceCharge inside a new variable and reuse
that instead of repeatedly indexing the map. It is always assumed that
the BalanceCharge exists in the map and is non-nil. A comment was
added as a reminder.

TotalCost cannot be calculated anymore from ChargingIncrement alone.
The function is not used, so it will not be causing any issues for
now.

BalanceCharge clone function now handles the case where it is nil
to avoid any possible nil pointer dereference.
2024-04-01 20:48:57 +03:00
ionutboangiu
a7ca167039 Add BalanceFactor value to Increments
For both the Increment and ChargingIncrement types.
2024-04-01 20:48:57 +03:00
ionutboangiu
bd4aa99458 Refactor balance's Factor field name with its plural form
Stored balances previous to this commit, that had a non-nil Factors map, will
now have Factors nil due to field name mismatch.
2024-04-01 19:58:04 +03:00
ionutboangiu
45066b2d2f Rename Factor field to Factors in BalanceSummaries 2024-04-01 19:58:04 +03:00
gezimbll
699989a336 removing cncReqs global variables 2024-03-30 16:36:30 +02:00
gezimbll
05e6e2660c updated CommitDate to fallback into default time 2024-03-25 15:09:00 +01:00
ionutboangiu
1e44d213ee Add integration build tags to stir shaken it test 2024-03-22 15:12:04 +01:00
gezimbll
cff5d22913 fixing unit tests on ersCfg 2024-03-13 18:26:01 +01:00
gezimbll
21d65b3810 added concurrent event processing for ers 2024-03-13 18:26:01 +01:00