Commit Graph

13916 Commits

Author SHA1 Message Date
ionutboangiu
c29f1fdd35 Remove services/cgr-engine.go
Trying to move services init before CGREngine constructor makes it pointless,
as it's only used as a dependency storage for services init.
2024-12-04 09:06:59 +01:00
DanB
2dfa1e7035 Adding initial stopChan to the engine 2024-12-01 20:22:47 +01:00
DanB
476f5ba877 Services with IntRPCConn method 2024-11-30 19:13:19 +01:00
DanB
6b241ee35b Adding serviceIndexer and StateDependencies 2024-11-28 14:56:03 +01:00
DanB
23e5f0e387 Add generic way to insert services inside services/CGREngine 2024-11-26 14:03:57 +01:00
DanB
76398441da Simplify running of the engine via only one Run command 2024-11-26 12:50:08 +01:00
DanB
3b195dcf1d Adding StateIndexer, ServiceIndexer, StateDeps 2024-11-23 20:14:07 +01:00
gezimbll
1c6c733a3f added dispatcher methods for rankings and trends 2024-11-22 20:29:27 +01:00
gezimbll
bb0116c543 added it test && loaders templates for ranking and trend profiles 2024-11-22 20:29:27 +01:00
armirveliaj
795d83325c Add coverage tests on engine 2024-11-14 17:48:40 +01:00
ionutboangiu
1453234293 Use channel instead of wait func (AnalyzerS) 2024-11-14 17:48:09 +01:00
ionutboangiu
59f8b1379e Use channel instead of wait func (CLS) 2024-11-14 17:48:09 +01:00
ionutboangiu
a1ed51b301 Comment services tests 2024-11-14 17:48:09 +01:00
ionutboangiu
21409fc92e Add new CommonListenerService 2024-11-14 17:48:09 +01:00
ionutboangiu
e78722ae4e Init cM,caps,cls,wg inside CGREngine constructor
Also pass the profile to the CGREngine struct the moment profiling started.
2024-11-14 17:48:09 +01:00
ionutboangiu
0d9358cf30 Ensure services wait for AnalyzerS to be initiated
Same logic as waiting for FilterS/DataDBService.
The difference will be that services will only wait when AnalyzerS
is enabled.
2024-11-14 17:48:09 +01:00
ionutboangiu
b7038dd61e Rename AnalyzerS constructor
NewAnalyzerService -> NewAnalyzerS
2024-11-14 17:48:09 +01:00
ionutboangiu
ab5328dcdb Name and document CommonListenerS mutex field 2024-11-14 17:48:09 +01:00
ionutboangiu
bb9bfbb19b Rename CommonListenerS constructor
NewServer -> NewCommonListenerS
2024-11-14 17:48:09 +01:00
ionutboangiu
4c8cf69f70 Auto-configure loader in test suite if needed 2024-11-07 15:38:37 +01:00
ionutboangiu
5bf8f1756c Extract cores.Server into new commonlisteners package
Functionality is the same.
Server has been renamed to CommonListenerS.
2024-11-04 21:23:15 +01:00
ionutboangiu
40c5d65af6 Add benchmark for diameter+caps
unusable until sessions is finished
2024-11-04 21:22:58 +01:00
ionutboangiu
709d70a031 Add DBCfg var for postgres 2024-11-04 21:22:58 +01:00
ionutboangiu
213b67fbb0 Revise registrarc integration test 2024-11-04 21:22:58 +01:00
ionutboangiu
113f1ccf31 Implement cgr-loader test helper 2024-11-04 21:22:58 +01:00
ionutboangiu
c498af5211 Allow benchmarks to use the testing suite 2024-11-04 21:22:58 +01:00
ionutboangiu
66c119dba7 Consider diameter requests when limiting caps
- add possibility to pass custom Error-Message AVP to negative diameter
  answers
- negative answer is now built only when an error occurs
- remove tests that were testing behaviour with max concurrent requests 0
- add integration test specific to this change
2024-11-04 21:22:58 +01:00
ionutboangiu
ad104573e9 Remove concurrent_requests diameter opt
Will be replaced by caps.
2024-11-04 21:22:58 +01:00
ionutboangiu
f937907b38 Add service toggle integration test 2024-11-04 21:22:58 +01:00
gezimbll
132a2b3bf9 rankings,trends: added implementation,services and tests 2024-11-04 19:03:21 +01:00
ionutboangiu
766a717b3c Add CI badge for integration tests 2024-11-03 13:19:45 +01:00
ionutboangiu
d42a1e0177 Fix annoying staticcheck unused param warnings 2024-11-03 13:19:45 +01:00
ionutboangiu
6cd07c5943 Port workflow changes from v0.11 2024-11-03 13:19:45 +01:00
ionutboangiu
d8acee74e2 Add *fileLineNumber var for file readers 2024-11-03 13:19:45 +01:00
ionutboangiu
c895681980 Improve CoreSv1.Status API with more detailed metrics
- add new metrics (off by default)
- add Debug boolean option for full metric details
- add Timezone option for 'running_since' field
- update default metrics shown and their constants
- remove utils.GetStartTime(), now use process metrics
2024-11-03 13:19:45 +01:00
ionutboangiu
176c91764f SplitPath: stop at '#' for nesting separator 2024-11-03 13:19:45 +01:00
ionutboangiu
bd79a0f9e7 Optimize removeFilterIndexesForFilter func (#4357)
Previously made two trips (get and set) for each group of items from
an index key being removed. Now, we fetch indexes once at the beginning
and store the updated indexes once at the end.
2024-11-03 13:19:45 +01:00
ionutboangiu
926dd4c4ec Update postgres driver version 2024-11-03 13:19:45 +01:00
ionutboangiu
0dfe689d8e Add postgres stor_db SSL opts 2024-11-03 13:19:45 +01:00
ionutboangiu
b2e25047bb Improve alignment inside config_defaults file 2024-11-01 15:59:39 +01:00
ionutboangiu
4b427aeaf6 Revise flag names/usage descriptions 2024-11-01 15:59:39 +01:00
ionutboangiu
78ae609f1d Simplify flag parsing by switching to ExitOnError
Will automatically handle 'ErrHelp' by exiting with status 0 when help is
requested and status 2 for other parsing errors.
2024-11-01 15:59:39 +01:00
ionutboangiu
792bbb3aff Add set_versions flag to cgr-engine
If set to true, it sets/overwrites all versions to the
current ones.
2024-11-01 15:59:39 +01:00
ionutboangiu
a251547408 Use correct consts for stordb types 2024-11-01 15:59:39 +01:00
ionutboangiu
0b48672df0 Improve readability of CheckVersions func
also removed extra spaces from errors
2024-11-01 15:59:39 +01:00
ionutboangiu
ce856c7815 Add http pprof_path cfg option
Remove redundant http_pprof cgr-engine flag since runtime profiling is now
configurable within the http cfg section and is enabled by default
2024-11-01 15:59:39 +01:00
ionutboangiu
c7dbcaea03 Revise CPU/Memory profiling
CPU profiling changes:

cgr-engine.go:
- use filepath.Join instead of path.Join
- handle *CoreService.StopCPUProfiling error inside deferred function
- same with the error from *os.File.Close()

cores/core.go:
- StartCPUProfile now returns an *os.File (as opposed to an io.WriteCloser),
  because os.File.Stat is used beforehand to check if a handler of the file is
  already active and confirm the status of profiling.  Asserting the type would
  have worked as well.
- handle pprof.StartCPUProfile error and ensure file is closed before returning
- log file close error as a warning if it occurs
- return missing mandatory error with correct path field name ('DirPath')
- no need to check if fileCPU is nil for profiling status
  - pprof.StartCPUProfiling will return an error if profiling is already started
  - os.File.Close() will return ErrClosed if profiling is already stopped
- differentiate between calling StopCPUProfiling when profiling hasn't started
and when it was already stopped by returning appropriate errors

Memory profiling changes:

- merge StopChanMemProf with StopMemoryProfiling
- remove fileMEM and stopMemProf from struct and constructors
- add separate mutex for memory profiling, ensure thread safety
- handle all significant errors
- log error if StopMemoryProfiling fails during CoreS Shutdown
- ignore errors if profiling inactive in Shutdown and deferred Stop
- move validations inside V1 functions
- return error if StartMemoryProfiling already started
- return error if StopMemoryProfiling already stopped or never started
- close profiling loop on error, not the cgr-engine
- StopMemoryProfiling closes channel and profiling loop writes final profile
- rename Path to DirPath for mandatory field error
- rename memprof_nrfiles flag to memprof_maxfiles
- increase default memprof_interval
- consider MaxFiles <= 0 as unlimited
- move memory profiling logic after starting services
- use CoreService Start/StopMemoryProfiling in main
- remove final memory profile block (created by deferred Stop)
- convert MemProfiling to method on CoreService and rename to profileMemory
- use Ticker for recurrent actions instead of Timer
- compute mem_final.prof full path in StartMemoryProfiling
- suffix profile files with current time instead of numbers
- update dispatcher methods after changes
- move MemoryPrf from utils to cores, rename to MemoryProfilingParams
- add logs for starting/stopping profiling
- added the possibility to disable timestamps in the memory profile file names
  and use increments of 1 instead.

Other changes:

- improved integration tests for flags (now table tests)
- improved profiling integration tests
2024-11-01 15:59:39 +01:00
ionutboangiu
e0c05ecfa7 Add redis pipeline window and limit cfg opts
Set redisPoolPipelineWindow to control duration before pipeline
flush (0 disables implicit pipelining) and redisPoolPipelineLimit
for max commands per pipeline (0 means no limit, only time window
applies).
2024-10-29 18:54:51 +01:00
ionutboangiu
3666dea54e Remove global CGRConfig used by tests 2024-10-29 18:54:51 +01:00
ionutboangiu
8cbeb45adf Remove global err used by tests
Removed many leftover error checks for NewDefaultCGRConfig,
which does not return an error anymore.
2024-10-29 18:54:51 +01:00