- add new metrics (off by default)
- add Debug boolean option for full metric details
- add Timezone option for 'running_since' field
- update default metrics shown and their constants
- remove utils.GetStartTime(), now use process metrics
CPU profiling changes:
cgr-engine.go:
- use filepath.Join instead of path.Join
- handle *CoreService.StopCPUProfiling error inside deferred function
- same with the error from *os.File.Close()
cores/core.go:
- StartCPUProfile now returns an *os.File (as opposed to an io.WriteCloser),
because os.File.Stat is used beforehand to check if a handler of the file is
already active and confirm the status of profiling. Asserting the type would
have worked as well.
- handle pprof.StartCPUProfile error and ensure file is closed before returning
- log file close error as a warning if it occurs
- return missing mandatory error with correct path field name ('DirPath')
- no need to check if fileCPU is nil for profiling status
- pprof.StartCPUProfiling will return an error if profiling is already started
- os.File.Close() will return ErrClosed if profiling is already stopped
- differentiate between calling StopCPUProfiling when profiling hasn't started
and when it was already stopped by returning appropriate errors
Memory profiling changes:
- merge StopChanMemProf with StopMemoryProfiling
- remove fileMEM and stopMemProf from struct and constructors
- add separate mutex for memory profiling, ensure thread safety
- handle all significant errors
- log error if StopMemoryProfiling fails during CoreS Shutdown
- ignore errors if profiling inactive in Shutdown and deferred Stop
- move validations inside V1 functions
- return error if StartMemoryProfiling already started
- return error if StopMemoryProfiling already stopped or never started
- close profiling loop on error, not the cgr-engine
- StopMemoryProfiling closes channel and profiling loop writes final profile
- rename Path to DirPath for mandatory field error
- rename memprof_nrfiles flag to memprof_maxfiles
- increase default memprof_interval
- consider MaxFiles <= 0 as unlimited
- move memory profiling logic after starting services
- use CoreService Start/StopMemoryProfiling in main
- remove final memory profile block (created by deferred Stop)
- convert MemProfiling to method on CoreService and rename to profileMemory
- use Ticker for recurrent actions instead of Timer
- compute mem_final.prof full path in StartMemoryProfiling
- suffix profile files with current time instead of numbers
- update dispatcher methods after changes
- move MemoryPrf from utils to cores, rename to MemoryProfilingParams
- add logs for starting/stopping profiling
- added the possibility to disable timestamps in the memory profile file names
and use increments of 1 instead.
Other changes:
- improved integration tests for flags (now table tests)
- improved profiling integration tests
Set redisPoolPipelineWindow to control duration before pipeline
flush (0 disables implicit pipelining) and redisPoolPipelineLimit
for max commands per pipeline (0 means no limit, only time window
applies).
Avoids the default 1 second delay when the batch doesn't
reach 100 messages within that time.
Useful when the Kafka exporter is not cached, as it would
otherwise encounter that delay. Setting BatchSize to 1
prevents this.
- made use of the test setup helpers.
- used t.Cleanup instead of defer.
- instead of waiting 50ms for the nats-server to start, used a helper
hook to attempt connections in fibonacci intervals. On success it
keeps a reference to the connection for later usage.
- handle error for stream delete function executed during cleanup.
- shorten time.Sleep durations when waiting for exports to finish.
- extract the cache itemID checking logic into a separate func
- retry failed requests in fibonacci intervals for up to 500ms
Make use of the previously added ees_success_ids and ees_failed_ids
configuration fields.
Remove Processed opts and everything related to them since they should
not be used anymore.
Fixed test compilation errors caused by the change.
Add ees_success_ids and ees_failed_ids fields in reader config. The
former will be used to set EeIDs when the event processing returns
no error, while the latter will be used otherwise (implementation
will follow).
Add config sanity checks for the added options.
Moved engine/cdr.go to utils/cdr.go.
Moved engine/cdrs.go and engine/cdrs_test.go to cdrs package.
Moved api functions from cdrs/cdrs.go to cdrs/apis.go. Same for the
tests.
Deleted CDRsV1 type and its methods.
Upgraded go.mod nats version due to an issue caused by version mismatch
between driver and server (uncertain).
Renamed function from getProcessOptions to getProcessedOptions.
## *NatsER.Serve
- Replaced ChanQueueSubscribe with QueueSubscribe for Core NATS consumer
to handle the message processing directly.
- Since QueueSubscribe is now used regardless of jetstream status, the
message handler has been assigned to a separate variable that can be
reused.
- The message handler is now dealing with the message processing
directly, therefore the select case listening for the channel which is
feeding NATS messages can be removed together with the channel itself
and the select. Currently, the goroutine within Serve only has to block
until the rdrExit chan is closed.
- Moved the resource check inside the handler right before starting the
message processing goroutine.
## *NatsEE.parseOpts
- Renamed function from parseOpt to parseOpts.
- Handled the error coming from GetNatsOpts function.
## *NatsEE.Connect
- Updated function to return early in case of non-nil nats.Conn value
to reduce nesting.
## *NatsEE.ExportEvent
- Use defer to release resources and RUnlock.
## *NatsEE.Close
- Use defer to Unlock.
- Update function to return early in case of nil nats.Conn value to
reduce nesting.
## ees.GetNatsOpts
- Chose switch over if else when parsing client certificate and keys
opts.
- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.
## ers.GetNatsOpts
- Chose switch over if else when parsing client certificate and keys
opts.
- Updated function to return the errors directly instead of assigning
them to a separate variable right before returning.
Removed tab from commented natsJetStreamMaxWaitProcessed option
value in config_defaults.go under ers section.
Added integration test for ERs NATS.
Updated ees/ers implementation to use the jetstream package which
separates the jetstream context from Core NATS.
Removed the jsOpts fields from the NatsEE struct. We are now using the
jetStreamMaxWait option directly through a timeout context.
Added streamName option for NATS reader since it is now required to be
specified when creating a consumer (it is not inferred based on subject
anymore).
Updated nats ers integration tests.
Updated tests to also use the new jetstream package.
Updated tests to start the nats-server using their official driver
instead of using the std go exec package. time.Sleeps are now not
required anymore to wait for the server.
In test configurations for nats readers, made sure that natsStreamName
option is populated. It is now required for consumers to know where to
subscribe.
Add StorDB to config and services.
Put back the store_cdrs option under cdrs and update the CDRs
service to depend on StorDB.
Define the StorDB interface and add a constructor for it.
Add a constructor for postgres storage.
Add a config sanity check to validate SSL modes for postgres.
Update cgr-engine to consider StorDB on startup.
