Commit Graph

588 Commits

Author SHA1 Message Date
ionutboangiu
c8a3ebe5e8 Use channel instead of context to handle shutdown 2025-01-10 20:18:08 +01:00
ionutboangiu
d9359a4005 Implement GuardianService and ConfigService
And use ServiceIndexer to sync with them
2025-01-10 20:18:08 +01:00
ionutboangiu
da2052e7b3 Implement FilterService
And use ServiceIndexer to sync with it
2024-12-12 11:55:14 +01:00
ionutboangiu
db301f7901 Remove StorDB service param from service constructors
Use ServiceIndexer instead.
Remove CDRs ListenAndServe method.
Pass the storDB connection directly to constructors instead of the
channel.
2024-12-12 11:55:14 +01:00
ionutboangiu
10dfcc5e48 Remove dmS parameter from service constructors
Use ServiceIndexer instead
2024-12-12 11:55:14 +01:00
ionutboangiu
55ecdf45e4 Remove clsChan in favor of using the service indexer 2024-12-12 11:55:14 +01:00
ionutboangiu
98b5b74b23 Remove cacheS parameter from service constructors
ServiceIndexer will be used instead
2024-12-12 11:55:14 +01:00
ionutboangiu
77af4f95b3 Remove anzCh in favor of using the service indexer 2024-12-10 18:53:13 +01:00
ionutboangiu
0783984bfe Remove srvDep from services that don't use it 2024-12-05 20:12:08 +01:00
ionutboangiu
79e461e7c5 Remove redundant internal channels
only from services that have dedicated files
2024-12-04 19:28:50 +01:00
ionutboangiu
cfdb3e80ca Integrate state deps across services for SERVICE_UP 2024-12-04 19:28:50 +01:00
ionutboangiu
c29f1fdd35 Remove services/cgr-engine.go
Trying to move services init before CGREngine constructor makes it pointless,
as it's only used as a dependency storage for services init.
2024-12-04 09:06:59 +01:00
DanB
2dfa1e7035 Adding initial stopChan to the engine 2024-12-01 20:22:47 +01:00
DanB
476f5ba877 Services with IntRPCConn method 2024-11-30 19:13:19 +01:00
DanB
6b241ee35b Adding serviceIndexer and StateDependencies 2024-11-28 14:56:03 +01:00
DanB
23e5f0e387 Add generic way to insert services inside services/CGREngine 2024-11-26 14:03:57 +01:00
DanB
76398441da Simplify running of the engine via only one Run command 2024-11-26 12:50:08 +01:00
DanB
3b195dcf1d Adding StateIndexer, ServiceIndexer, StateDeps 2024-11-23 20:14:07 +01:00
gezimbll
bb0116c543 added it test && loaders templates for ranking and trend profiles 2024-11-22 20:29:27 +01:00
ionutboangiu
1453234293 Use channel instead of wait func (AnalyzerS) 2024-11-14 17:48:09 +01:00
ionutboangiu
59f8b1379e Use channel instead of wait func (CLS) 2024-11-14 17:48:09 +01:00
ionutboangiu
a1ed51b301 Comment services tests 2024-11-14 17:48:09 +01:00
ionutboangiu
21409fc92e Add new CommonListenerService 2024-11-14 17:48:09 +01:00
ionutboangiu
e78722ae4e Init cM,caps,cls,wg inside CGREngine constructor
Also pass the profile to the CGREngine struct the moment profiling started.
2024-11-14 17:48:09 +01:00
ionutboangiu
0d9358cf30 Ensure services wait for AnalyzerS to be initiated
Same logic as waiting for FilterS/DataDBService.
The difference will be that services will only wait when AnalyzerS
is enabled.
2024-11-14 17:48:09 +01:00
ionutboangiu
b7038dd61e Rename AnalyzerS constructor
NewAnalyzerService -> NewAnalyzerS
2024-11-14 17:48:09 +01:00
ionutboangiu
bb9bfbb19b Rename CommonListenerS constructor
NewServer -> NewCommonListenerS
2024-11-14 17:48:09 +01:00
ionutboangiu
5bf8f1756c Extract cores.Server into new commonlisteners package
Functionality is the same.
Server has been renamed to CommonListenerS.
2024-11-04 21:23:15 +01:00
ionutboangiu
213b67fbb0 Revise registrarc integration test 2024-11-04 21:22:58 +01:00
ionutboangiu
66c119dba7 Consider diameter requests when limiting caps
- add possibility to pass custom Error-Message AVP to negative diameter
  answers
- negative answer is now built only when an error occurs
- remove tests that were testing behaviour with max concurrent requests 0
- add integration test specific to this change
2024-11-04 21:22:58 +01:00
gezimbll
132a2b3bf9 rankings,trends: added implementation,services and tests 2024-11-04 19:03:21 +01:00
ionutboangiu
0dfe689d8e Add postgres stor_db SSL opts 2024-11-03 13:19:45 +01:00
ionutboangiu
4b427aeaf6 Revise flag names/usage descriptions 2024-11-01 15:59:39 +01:00
ionutboangiu
78ae609f1d Simplify flag parsing by switching to ExitOnError
Will automatically handle 'ErrHelp' by exiting with status 0 when help is
requested and status 2 for other parsing errors.
2024-11-01 15:59:39 +01:00
ionutboangiu
792bbb3aff Add set_versions flag to cgr-engine
If set to true, it sets/overwrites all versions to the
current ones.
2024-11-01 15:59:39 +01:00
ionutboangiu
a251547408 Use correct consts for stordb types 2024-11-01 15:59:39 +01:00
ionutboangiu
ce856c7815 Add http pprof_path cfg option
Remove redundant http_pprof cgr-engine flag since runtime profiling is now
configurable within the http cfg section and is enabled by default
2024-11-01 15:59:39 +01:00
ionutboangiu
c7dbcaea03 Revise CPU/Memory profiling
CPU profiling changes:

cgr-engine.go:
- use filepath.Join instead of path.Join
- handle *CoreService.StopCPUProfiling error inside deferred function
- same with the error from *os.File.Close()

cores/core.go:
- StartCPUProfile now returns an *os.File (as opposed to an io.WriteCloser),
  because os.File.Stat is used beforehand to check if a handler of the file is
  already active and confirm the status of profiling.  Asserting the type would
  have worked as well.
- handle pprof.StartCPUProfile error and ensure file is closed before returning
- log file close error as a warning if it occurs
- return missing mandatory error with correct path field name ('DirPath')
- no need to check if fileCPU is nil for profiling status
  - pprof.StartCPUProfiling will return an error if profiling is already started
  - os.File.Close() will return ErrClosed if profiling is already stopped
- differentiate between calling StopCPUProfiling when profiling hasn't started
and when it was already stopped by returning appropriate errors

Memory profiling changes:

- merge StopChanMemProf with StopMemoryProfiling
- remove fileMEM and stopMemProf from struct and constructors
- add separate mutex for memory profiling, ensure thread safety
- handle all significant errors
- log error if StopMemoryProfiling fails during CoreS Shutdown
- ignore errors if profiling inactive in Shutdown and deferred Stop
- move validations inside V1 functions
- return error if StartMemoryProfiling already started
- return error if StopMemoryProfiling already stopped or never started
- close profiling loop on error, not the cgr-engine
- StopMemoryProfiling closes channel and profiling loop writes final profile
- rename Path to DirPath for mandatory field error
- rename memprof_nrfiles flag to memprof_maxfiles
- increase default memprof_interval
- consider MaxFiles <= 0 as unlimited
- move memory profiling logic after starting services
- use CoreService Start/StopMemoryProfiling in main
- remove final memory profile block (created by deferred Stop)
- convert MemProfiling to method on CoreService and rename to profileMemory
- use Ticker for recurrent actions instead of Timer
- compute mem_final.prof full path in StartMemoryProfiling
- suffix profile files with current time instead of numbers
- update dispatcher methods after changes
- move MemoryPrf from utils to cores, rename to MemoryProfilingParams
- add logs for starting/stopping profiling
- added the possibility to disable timestamps in the memory profile file names
  and use increments of 1 instead.

Other changes:

- improved integration tests for flags (now table tests)
- improved profiling integration tests
2024-11-01 15:59:39 +01:00
ionutboangiu
e0c05ecfa7 Add redis pipeline window and limit cfg opts
Set redisPoolPipelineWindow to control duration before pipeline
flush (0 disables implicit pipelining) and redisPoolPipelineLimit
for max commands per pipeline (0 means no limit, only time window
applies).
2024-10-29 18:54:51 +01:00
ionutboangiu
f11efbad51 Add precache support for exporters
Additional changes:
- removed unnecessary locking when initializing the exporter
  cache map, as the service itself is responsible for locking.
- separated setupCache method into ClearExporterCache and
  SetupExporterCache methods.
- removed idle ListenAndServe function that was only waiting for the
  stopChan to close. The reload case was unreachable due to the reload
  channel being created in Start instead of being passed down.
- removed Shutdown method on EventExporterS and replaced it with the
  exported ClearExporterCache method as it provided the same
  functionality.
2024-10-29 18:54:51 +01:00
armirveliaj
0b2367c934 Add new unit tests for rankings 2024-10-17 16:34:33 +02:00
armirveliaj
5076e6f668 Add new unit tests on TrendS 2024-10-14 15:42:54 +02:00
ionutboangiu
65e8128303 Implement ErSv1.RunReader api 2024-09-23 20:28:26 +02:00
ionutboangiu
da8c468c0c Remove unused parameters from fs connect func 2024-09-23 20:28:26 +02:00
ionutboangiu
449bc5c08a Add active_session_delimiter option for fs_agent
Used for 'show_channels' requests and responses.
2024-09-23 20:28:26 +02:00
gezimblliku
0f81b78321 updated trend profile && trends service implementation 2024-09-23 11:48:04 +02:00
ionutboangiu
1c73e42307 Revise ERs event exporting
Make use of the previously added ees_success_ids and ees_failed_ids
configuration fields.

Remove Processed opts and everything related to them since they should
not be used anymore.

Fixed test compilation errors caused by the change.
2024-09-05 20:46:40 +02:00
ionutboangiu
7c5d56c1de Combine csv loader tests
- ensures the tpreader is not eating memory even when tests are
not being run (before it was being constructed in an init())
2024-08-15 18:17:33 +02:00
gezimblliku
73a7590f1c added profiles,configs and services for trend && ranking 2024-08-02 09:37:52 +02:00
Gezim Blliku
80249e1074 implemented janusagent & added to services 2024-07-21 19:07:43 +02:00