Commit Graph

1577 Commits

Author SHA1 Message Date
ionutboangiu
da2052e7b3 Implement FilterService
And use ServiceIndexer to sync with it
2024-12-12 11:55:14 +01:00
ionutboangiu
db301f7901 Remove StorDB service param from service constructors
Use ServiceIndexer instead.
Remove CDRs ListenAndServe method.
Pass the storDB connection directly to constructors instead of the
channel.
2024-12-12 11:55:14 +01:00
ionutboangiu
10dfcc5e48 Remove dmS parameter from service constructors
Use ServiceIndexer instead
2024-12-12 11:55:14 +01:00
ionutboangiu
55ecdf45e4 Remove clsChan in favor of using the service indexer 2024-12-12 11:55:14 +01:00
ionutboangiu
98b5b74b23 Remove cacheS parameter from service constructors
ServiceIndexer will be used instead
2024-12-12 11:55:14 +01:00
ionutboangiu
77af4f95b3 Remove anzCh in favor of using the service indexer 2024-12-10 18:53:13 +01:00
ionutboangiu
0783984bfe Remove srvDep from services that don't use it 2024-12-05 20:12:08 +01:00
ionutboangiu
34de7e2c4e Remove unused dependency waitgroups
to declutter cgr-engine.go a bit. Entire map will be removed in the future
2024-12-04 19:28:50 +01:00
ionutboangiu
79e461e7c5 Remove redundant internal channels
only from services that have dedicated files
2024-12-04 19:28:50 +01:00
ionutboangiu
c29f1fdd35 Remove services/cgr-engine.go
Trying to move services init before CGREngine constructor makes it pointless,
as it's only used as a dependency storage for services init.
2024-12-04 09:06:59 +01:00
DanB
2dfa1e7035 Adding initial stopChan to the engine 2024-12-01 20:22:47 +01:00
DanB
6b241ee35b Adding serviceIndexer and StateDependencies 2024-11-28 14:56:03 +01:00
DanB
23e5f0e387 Add generic way to insert services inside services/CGREngine 2024-11-26 14:03:57 +01:00
DanB
76398441da Simplify running of the engine via only one Run command 2024-11-26 12:50:08 +01:00
ionutboangiu
e78722ae4e Init cM,caps,cls,wg inside CGREngine constructor
Also pass the profile to the CGREngine struct the moment profiling started.
2024-11-14 17:48:09 +01:00
ionutboangiu
bb9bfbb19b Rename CommonListenerS constructor
NewServer -> NewCommonListenerS
2024-11-14 17:48:09 +01:00
ionutboangiu
5bf8f1756c Extract cores.Server into new commonlisteners package
Functionality is the same.
Server has been renamed to CommonListenerS.
2024-11-04 21:23:15 +01:00
ionutboangiu
4b427aeaf6 Revise flag names/usage descriptions 2024-11-01 15:59:39 +01:00
ionutboangiu
78ae609f1d Simplify flag parsing by switching to ExitOnError
Will automatically handle 'ErrHelp' by exiting with status 0 when help is
requested and status 2 for other parsing errors.
2024-11-01 15:59:39 +01:00
ionutboangiu
c7dbcaea03 Revise CPU/Memory profiling
CPU profiling changes:

cgr-engine.go:
- use filepath.Join instead of path.Join
- handle *CoreService.StopCPUProfiling error inside deferred function
- same with the error from *os.File.Close()

cores/core.go:
- StartCPUProfile now returns an *os.File (as opposed to an io.WriteCloser),
  because os.File.Stat is used beforehand to check if a handler of the file is
  already active and confirm the status of profiling.  Asserting the type would
  have worked as well.
- handle pprof.StartCPUProfile error and ensure file is closed before returning
- log file close error as a warning if it occurs
- return missing mandatory error with correct path field name ('DirPath')
- no need to check if fileCPU is nil for profiling status
  - pprof.StartCPUProfiling will return an error if profiling is already started
  - os.File.Close() will return ErrClosed if profiling is already stopped
- differentiate between calling StopCPUProfiling when profiling hasn't started
and when it was already stopped by returning appropriate errors

Memory profiling changes:

- merge StopChanMemProf with StopMemoryProfiling
- remove fileMEM and stopMemProf from struct and constructors
- add separate mutex for memory profiling, ensure thread safety
- handle all significant errors
- log error if StopMemoryProfiling fails during CoreS Shutdown
- ignore errors if profiling inactive in Shutdown and deferred Stop
- move validations inside V1 functions
- return error if StartMemoryProfiling already started
- return error if StopMemoryProfiling already stopped or never started
- close profiling loop on error, not the cgr-engine
- StopMemoryProfiling closes channel and profiling loop writes final profile
- rename Path to DirPath for mandatory field error
- rename memprof_nrfiles flag to memprof_maxfiles
- increase default memprof_interval
- consider MaxFiles <= 0 as unlimited
- move memory profiling logic after starting services
- use CoreService Start/StopMemoryProfiling in main
- remove final memory profile block (created by deferred Stop)
- convert MemProfiling to method on CoreService and rename to profileMemory
- use Ticker for recurrent actions instead of Timer
- compute mem_final.prof full path in StartMemoryProfiling
- suffix profile files with current time instead of numbers
- update dispatcher methods after changes
- move MemoryPrf from utils to cores, rename to MemoryProfilingParams
- add logs for starting/stopping profiling
- added the possibility to disable timestamps in the memory profile file names
  and use increments of 1 instead.

Other changes:

- improved integration tests for flags (now table tests)
- improved profiling integration tests
2024-11-01 15:59:39 +01:00
ionutboangiu
e0c05ecfa7 Add redis pipeline window and limit cfg opts
Set redisPoolPipelineWindow to control duration before pipeline
flush (0 disables implicit pipelining) and redisPoolPipelineLimit
for max commands per pipeline (0 means no limit, only time window
applies).
2024-10-29 18:54:51 +01:00
ionutboangiu
3e98e37db3 Revise NewRPCClient helper function 2024-10-23 21:11:25 +02:00
ionutboangiu
0063ef3822 Add support for mongodb+srv conn format
Option can be found in db opts under the name 'mongoConnScheme'.
2024-09-05 20:46:40 +02:00
ionutboangiu
169b5500d3 Revise err handling for CSV storage constructor
NewFileCSVStorage() now returns an error besides the storage struct itself, which is
logged and returned instead of calling log.Fatal() which was causing the engine to
crash.

Fixed compilation errors by creating the CSVStorage separately and passing it as an
argument to the TpReader constructor.

Fixes #3962
2024-09-05 20:46:40 +02:00
ionutboangiu
1b24a504dc add flaky build constraint to unstable tests 2024-08-21 20:59:10 +02:00
ionutboangiu
0f70af84e9 centralize test flags for shared use across tests 2024-08-15 18:17:33 +02:00
ionutboangiu
0dfb3b860d Centralize RPCClient constructor 2024-08-15 18:17:33 +02:00
armirveliaj
6dc612a32a Add print_config flags to cgr-loader and cgr-migrator 2024-07-03 10:39:59 +02:00
gezimbll
3ff0e6e5a1 revising integration tests 2024-06-07 17:18:23 +02:00
arberkatellari
de5db4a938 Add caching_delay to config and loaders 2024-03-06 16:16:08 +01:00
ionutboangiu
113e2a2bdf Replace interface{} with any 2023-05-31 10:22:28 +02:00
gezimbll
04d05c3a4f replacing non-meta constants for storage types 2023-03-08 19:13:48 +01:00
DanB
8d37467b63 Reorganising dispatcherS routeID functionality to avoid extra processing of event 2022-08-27 15:53:48 +02:00
ionutboangiu
0653e9b5d7 Apply go fmt 2022-08-24 09:32:23 +03:00
adi
424b5bde54 Improved service for efs 2022-08-23 20:31:53 +03:00
ionutboangiu
4857245aa5 Use copy instead of loop + other staticcheck fixes 2022-07-15 09:28:57 +02:00
ionutboangiu
d9f567b0a3 Rename RedisMaxAttempts to RedisConnectionAttempts 2022-07-14 14:59:28 +02:00
ionutboangiu
52f19d3a73 Repair it tests 2022-07-14 14:59:28 +02:00
ionutboangiu
81793af5de Make hardcoded options configurable for redis and solve compilation errors 2022-07-07 18:05:14 +02:00
ionutboangiu
523045cc12 Add timeout options for redis and solve compilation errors 2022-07-03 13:08:52 +02:00
ionutboangiu
cc467eb484 Update tests 2022-07-03 13:08:52 +02:00
ionutboangiu
5437b3a8a2 Repair compilation errors after library updates 2022-07-03 13:08:52 +02:00
ionutboangiu
e98f230f04 Change type name from Blockers to DynamicBlockers 2022-05-10 17:05:31 +02:00
ionutboangiu
52c79ca8db Change RouteBlockers type from bool to DynamicBlockers 2022-04-29 20:33:51 +02:00
porosnicuadrian
41a7ee8add DynamicBlocker for stats + unit test 2022-04-27 17:49:14 +02:00
andronache98
ecdf5e54ea Changed headers 2022-03-09 14:14:30 +01:00
andronache98
9ec56f5155 Changed headers 2022-03-09 14:14:30 +01:00
andronache98
74d8a61c15 Repaired most integration tests using storDB 2022-03-09 14:14:30 +01:00
andronache98
928dbd9e42 Removing StorDB 2022-03-09 14:14:30 +01:00
andronache98
03793c0082 Removed storDB from config 2022-03-09 14:14:30 +01:00