Updating architecture docs structure

This commit is contained in:
DanB
2020-01-14 20:03:20 +01:00
parent 3e632ee1a9
commit e80b656b8d
8 changed files with 314 additions and 516 deletions

View File

@@ -1,519 +1,16 @@
2. Architecture
===============
The CGRateS framework consists of functionality packed within **five** software applications, described below.
.. hlist::
:columns: 5
- `cgr-engine`_
- `cgr-console`_
- `cgr-loader`_
- `cgr-migrator`_
- `cgr-tester`_
.. _cgr-engine:
2.1. cgr-engine
---------------
.. figure:: images/CGRateSInternalArchitecture.png
:alt: CGRateS Internal Architecture
:align: Center
:scale: 75 %
Internal Architecture of **cgr-engine**_.
Main component, implements most of the CGRateS functionality in form of services customisable through the use of *json* engine_configuration_ or command line arguments.
::
cgrates@OCS:~$ cgr-engine -help
Usage of cgr-engine:
-cdrs
Enforce starting of the cdrs daemon overwriting config
-config_path string
Configuration directory path. (default "/etc/cgrates/")
-cpuprofile string
write cpu profile to file
-pid string
Write pid file
-rater
Enforce starting of the rater daemon overwriting config
-scheduler
Enforce starting of the scheduler daemon .overwriting config
-scheduled_shutdown string
shutdown the engine after this duration
-singlecpu
Run on single CPU core
-version
Prints the application version.
.. hint:: # cgr-engine -config_path=/etc/cgrates
2.1.1. RALs service
~~~~~~~~~~~~~~~~~~~~
Responsible with the following tasks:
- Operates on balances.
- Computes prices for rating subjects.
- Monitors and executes triggers.
- LCR functionality
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (dataDb)
"stor_db" - (cdrDb, loadDb)
- Config section in the CGRateS configuration file:
- ``"rals": {...}``
2.1.2. Scheduler service
~~~~~~~~~~~~~~~~~~~~~~~~
Used to execute periodic/scheduled tasks.
- Communicates via:
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (dataDb)
- Config section in the CGRateS configuration file:
- ``"schedulers": {...}``
2.1.3. SessionManager service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Responsible with call control on the Telecommunication Switch side. Operates in two different modes (per call or globally):
- PREPAID
- Monitors call start.
- Checks balance availability for the call.
- Enforces global timer for a call at call-start.
- Executes routing commands for the call where that is necessary ( eg call un-park in case of FreeSWITCH).
- Periodically executes balance debits on call at the beginning of debit interval.
- Enforce call disconnection on insufficient balance.
- Refunds the balance taken in advance at the call stop.
- POSTPAID
- Executes balance debit on call-stop.
All call actions are logged into CGRateS's LogDB.
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"stor_db" - (cdrDb)
2.1.4. DiameterAgent service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Responsible for the communication with Diameter server via diameter protocol.
Despite the name it is a flexible **Diameter Server**.
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
- none
- Config section in the CGRateS configuration file:
- ``"diameter_agent": {...}``
2.1.5. CDR service
~~~~~~~~~~~~~~~~~~~
Centralized CDR server and CDR (raw or rated) **replicator**.
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"stor_db" - (cdrDb)
"data_db" - (accountDb)
- Config section in the CGRateS configuration file:
- ``"cdrs": {...}``
2.1.6. CDRStats service
~~~~~~~~~~~~~~~~~~~~~~~
Computes real-time CDR stats. Capable with real-time fraud detection and mitigation with actions triggered.
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (dataDb)
- Config section in the CGRateS configuration file:
- ``"cdrstats": {...}``
2.1.7. CDRC service
~~~~~~~~~~~~~~~~~~~
Gathers offline CDRs and post them to CDR Server - (CDRS component)
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
- none
- Config section in the CGRateS configuration file:
- ``"cdrc": {...}``
2.1.8. Aliases service
~~~~~~~~~~~~~~~~~~~~~~~
Generic purpose **aliasing** system.
Possible applications:
- Change destination name based on user or destination prefix matched.
- Change lcr supplier name based on the user calling.
- Locale specifics, ability to display specific tags in user defined language.
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (accountDb)
- Config section in the CGRateS configuration file:
- ``"aliases": {...}``
2.1.9. User service
~~~~~~~~~~~~~~~~~~~~
Generic purpose **user** system to maintain user profiles (LDAP similarity).
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (accountDb)
- Config section in the CGRateS configuration file:
- ``"users": {...}``
2.1.10. PubSub service
~~~~~~~~~~~~~~~~~~~~~~
PubSub service used to expose internal events to interested external components (eg: balance ops)
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (accountDb)
- Config section in the CGRateS configuration file:
- ``"pubsubs": {...}``
2.1.11. Resource Limiter service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Resource Limiter service used to limit resources during authorization (eg: maximum calls per destination for an account)
- Communicates via:
- RPC
- internal/in-process *within the same running* **cgr-engine** process.
- Operates with the following CGRateS database(s): ::
"data_db" - (accountDb)
- Config section in the CGRateS configuration file:
- ``"rls": {...}``
2.1.12. APIER RPC service
~~~~~~~~~~~~~~~~~~~~~~~~~
RPC service used to expose external access towards internal components.
- Communicates via:
- JSON/GOB over socket
- JSON over HTTP
- JSON over WebSocket
2.1.13. Cdre
~~~~~~~~~~~~
Component to retrieve rated CDRs from internal CDRs database.
- Communicates via:
- Operates with the following CGRateS database(s): ::
"stor_db" - (cdrDb)
- Config section in the CGRateS configuration file:
- ``"cdre": {...}``
2.1.14. Mailer
~~~~~~~~~~~~~~
TBD
- Communicates via:
- Operates with the following CGRateS database(s):
- Config section in the CGRateS configuration file:
- ``"mailer": {...}``
2.1.15. Suretax
~~~~~~~~~~~~~~~
TBD
- Communicates via:
- Operates with the following CGRateS database(s):
- Config section in the CGRateS configuration file:
- ``"suretax": {...}``
2.1.X Mediator service
~~~~~~~~~~~~~~~~~~~~~~
.. important:: This service is not valid anymore. Its functionality is replaced by CDRC and CDRS services.
Responsible to mediate the CDRs generated by Telecommunication Switch.
Has the ability to combine CDR fields into rating subject and run multiple mediation processes on the same record.
On Linux machines, able to work with inotify kernel subsystem in order to process the records close to real-time after the Switch has released them.
2.2. cgr-loader
---------------
Used for importing the rating information into the CGRateS database system.
Can be used to:
- Import information from **csv files** to **data_db**.
- Import information from **csv files** to **stor_db**. ``-to_stordb -tpid``
- Import information from **stor_db** to **data_db**. ``-from_stordb -tpid``
::
cgrates@OCS:~$ cgr-loader -help
Usage of cgr-loader:
-cdrstats_address string
CDRStats service to contact for data reloads, empty to disable automatic data reloads (default "127.0.0.1:2013")
-datadb_host string
The DataDb host to connect to. (default "127.0.0.1")
-datadb_name string
The name/number of the DataDb to connect to. (default "11")
-datadb_passwd string
The DataDb user's password.
-datadb_port string
The DataDb port to bind to. (default "6379")
-datadb_type string
The type of the DataDb database <redis> (default "redis")
-datadb_user string
The DataDb user to sign in as.
-dbdata_encoding string
The encoding used to store object data in strings (default "msgpack")
-disable_reverse_mappings
Will disable reverse mappings rebuilding
-dry_run
When true will not save loaded data to dataDb but just parse it for consistency and errors.
-flushdb
Flush the database before importing
-from_stordb
Load the tariff plan from storDb to dataDb
-migrate_rc8 string
Migrate Accounts, Actions, ActionTriggers, DerivedChargers, ActionPlans and SharedGroups to RC8 structures, possible values: *all,acc,atr,act,dcs,apl,shg
-path string
The path to folder containing the data files (default "./")
-rater_address string
Rater service to contact for cache reloads, empty to disable automatic cache reloads (default "127.0.0.1:2013")
-runid string
Uniquely identify an import/load, postpended to some automatic fields
-stats
Generates statsistics about given data.
-stordb_host string
The storDb host to connect to. (default "127.0.0.1")
-stordb_name string
The name/number of the storDb to connect to. (default "cgrates")
-stordb_passwd string
The storDb user's password. (default "CGRateS.org")
-stordb_port string
The storDb port to bind to. (default "3306")
-stordb_type string
The type of the storDb database <mysql> (default "mysql")
-stordb_user string
The storDb user to sign in as. (default "cgrates")
-timezone string
Timezone for timestamps where not specified <""|UTC|Local|$IANA_TZ_DB> (default "Local")
-to_stordb
Import the tariff plan from files to storDb
-validate
When true will run various check on the loaded data to check for structural errors
-verbose
Enable detailed verbose logging output
-version
Prints the application version.
.. hint:: # cgr-loader -flushdb
.. hint:: # cgr-loader -verbose -datadb_port="27017" -datadb_type="mongo"
2.3. cgr-console
----------------
Command line tool used to interface with the RALs service. Able to execute **sub-commands**.
::
cgrates@OCS:~$ cgr-console -help
Usage of cgr-console:
-rpc_encoding string
RPC encoding used <gob|json> (default "json")
-server string
server address host:port (default "127.0.0.1:2012")
-verbose
Show extra info about command execution.
-version
Prints the application version.
rif@grace:~$ cgr-console help_more
2013/04/13 17:23:51
Usage: cgr-console [cfg_opts...{-h}] <status|get_balance>
.. hint:: # cgr-console status
2.4. cgr-tester
---------------
Command line stress testing tool.
::
cgrates@OCS:~$ cgr-tester --help
Usage of cgr-tester:
-datadb_host string
The DataDb host to connect to. (default "127.0.0.1")
-datadb_name string
The name/number of the DataDb to connect to. (default "11")
-datatdb_passwd string
The DataDb user's password.
-datadb_port string
The DataDb port to bind to. (default "6379")
-datadb_type string
The type of the DataDb database <redis> (default "redis")
-datadb_user string
The DataDb user to sign in as.
-category string
The Record category to test. (default "call")
-cpuprofile string
write cpu profile to file
-dbdata_encoding string
The encoding used to store object data in strings. (default "msgpack")
-destination string
The destination to use in queries. (default "1002")
-json
Use JSON RPC
-memprofile string
write memory profile to this file
-parallel int
run n requests in parallel
-rater_address string
Rater address for remote tests. Empty for internal rater.
-runs int
stress cycle number (default 10000)
-subject string
The rating subject to use in queries. (default "1001")
-tenant string
The type of record to use in queries. (default "cgrates.org")
-tor string
The type of record to use in queries. (default "*voice")
.. hint:: # cgr-tester -runs=10000
2.5. cgr-migrator
-----------------
Command line migration tool.
::
cgrates@OCS:~$ cgr-migrator --help
Usage of cgr-migrator:
-datadb_host string
The DataDb host to connect to. (default "192.168.100.40")
-datadb_name string
The name/number of the DataDb to connect to. (default "10")
-datadb_passwd string
The DataDb user's password.
-datadb_port string
The DataDb port to bind to. (default "6379")
-datadb_type string
The type of the DataDb database <redis> (default "redis")
-datadb_user string
The DataDb user to sign in as. (default "cgrates")
-dbdata_encoding string
The encoding used to store object data in strings (default "msgpack")
-dry_run
When true will not save loaded data to dataDb but just parse it for consistency and errors.(default "false")
-migrate string
Fire up automatic migration *to use multiple values use ',' as separator
<*set_versions|*cost_details|*accounts|*actions|*action_triggers|*action_plans|*shared_groups>
-old_datadb_host string
The DataDb host to connect to. (default "192.168.100.40")
-old_datadb_name string
The name/number of the DataDb to connect to. (default "10")
-old_datadb_passwd string
The DataDb user's password.
-old_datadb_port string
The DataDb port to bind to. (default "6379")
-old_datadb_type string
The type of the DataDb database <redis>
-old_datadb_user string
The DataDb user to sign in as. (default "cgrates")
-old_dbdata_encoding string
The encoding used to store object data in strings
-old_stordb_host string
The storDb host to connect to. (default "192.168.100.40")
-old_stordb_name string
The name/number of the storDb to connect to. (default "cgrates")
-old_stordb_passwd string
The storDb user's password.
-old_stordb_port string
The storDb port to bind to. (default "3306")
-old_stordb_type string
The type of the storDb database <mysql|postgres>
-old_stordb_user string
The storDb user to sign in as. (default "cgrates")
-stats
Generates statsistics about given data.(default "false")
-stordb_host string
The storDb host to connect to. (default "192.168.100.40")
-stordb_name string
The name/number of the storDb to connect to. (default "cgrates")
-stordb_passwd string
The storDb user's password.
-stordb_port string
The storDb port to bind to. (default "3306")
-stordb_type string
The type of the storDb database <mysql|postgres> (default "mysql")
-stordb_user string
The storDb user to sign in as. (default "cgrates")
-verbose
Enable detailed verbose logging output.(default "false")
-version
Prints the application version.
.. toctree::
:maxdepth: 2
cgr-engine
cgr-console
cgr-loader
cgr-migrator
cgr-tester

31
docs/cgr-console.rst Normal file
View File

@@ -0,0 +1,31 @@
.. _cgr-console:
2.2. cgr-console
----------------
Command line tool used to interface with the APIs implemented within `cgr-engine`_.
Configurable via command line arguments.
::
$ cgr-console -help
Usage of cgr-console:
-ca_path string
path to CA for tls connection(only for self sign certificate)
-crt_path string
path to certificate for tls connection
-key_path string
path to key for tls connection
-reply_timeout int
Reply timeout in seconds (default 300)
-rpc_encoding string
RPC encoding used <*gob|*json> (default "*json")
-server string
server address host:port (default "127.0.0.1:2012")
-tls
TLS connection
-verbose
Show extra info about command execution.
-version
Prints the application version.
.. hint:: # cgr-console status

47
docs/cgr-engine.rst Normal file
View File

@@ -0,0 +1,47 @@
.. _cgr-engine:
2.1. cgr-engine
---------------
.. figure:: images/CGRateSInternalArchitecture.png
:alt: CGRateS Internal Architecture
:align: Center
:scale: 75 %
Internal Architecture of **cgr-engine**
Groups various services and components, customisable through the use of *json* engine_configuration_ or command line arguments (with higher prio).
::
$ cgr-engine -help
Usage of cgr-engine:
-config_path string
Configuration directory path. (default "/etc/cgrates/")
-cpuprof_dir string
write cpu profile to files
-httprof_path string
http address used for program profiling
-log_level int
Log level (0-emergency to 7-debug) (default -1)
-logger string
logger <*syslog|*stdout>
-memprof_dir string
write memory profile to file
-memprof_interval duration
Time betwen memory profile saves (default 5s)
-memprof_nrfiles int
Number of memory profile to write (default 1)
-node_id string
The node ID of the engine
-pid string
Write pid file
-scheduled_shutdown string
shutdown the engine after this duration
-singlecpu
Run on single CPU core
-version
Prints the application version.
.. hint:: # cgr-engine -config_path=/etc/cgrates

87
docs/cgr-loader.rst Normal file
View File

@@ -0,0 +1,87 @@
.. _cgr-loader:
2.3. cgr-loader
---------------
Tool used to load/import TariffPlan data into CGRateS databases.
Can be used to:
* load TariffPlan data from **csv files** to **DataDB**.
* import TariffPlan data from **csv files** to **StorDB** as offline data. ``-to_stordb -tpid``
* import TariffPlan data from **StorDB** to **DataDB**. ``-from_stordb -tpid``
Configurable via engine_configuration_ or command line arguments (with higher prio).
::
$ cgr-loader -h
Usage of cgr-loader:
-api_key string
Api Key used to comosed ArgDispatcher
-caches_address string
CacheS component to contact for cache reloads, empty to disable automatic cache reloads (default "*localhost")
-caching string
Caching strategy used when loading TP
-config_path string
Configuration directory path.
-datadb_host string
The DataDb host to connect to. (default "127.0.0.1")
-datadb_name string
The name/number of the DataDb to connect to. (default "10")
-datadb_passwd string
The DataDb user's password.
-datadb_port string
The DataDb port to bind to. (default "6379")
-datadb_type string
The type of the DataDB database <*redis|*mongo> (default "redis")
-datadb_user string
The DataDb user to sign in as. (default "cgrates")
-dbdata_encoding string
The encoding used to store object data in strings (default "msgpack")
-disable_reverse_mappings
Will disable reverse mappings rebuilding
-dry_run
When true will not save loaded data to dataDb but just parse it for consistency and errors.
-field_sep string
Separator for csv file (by default "," is used) (default ",")
-flush_stordb
Remove tariff plan data for id from the database
-from_stordb
Load the tariff plan from storDb to dataDb
-import_id string
Uniquely identify an import/load, postpended to some automatic fields
-path string
The path to folder containing the data files (default "./")
-recursive
Loads data from folder recursive.
-redis_sentinel string
The name of redis sentinel
-remove
Will remove instead of adding data from DB
-route_id string
RouteID used to comosed ArgDispatcher
-rpc_encoding string
RPC encoding used <*gob|*json> (default "*json")
-scheduler_address string
(default "*localhost")
-stordb_host string
The storDb host to connect to. (default "127.0.0.1")
-stordb_name string
The name/number of the storDb to connect to. (default "cgrates")
-stordb_passwd string
The storDb user's password.
-stordb_port string
The storDb port to bind to. (default "3306")
-stordb_type string
The type of the storDb database <*mysql|*postgres|*mongo> (default "mysql")
-stordb_user string
The storDb user to sign in as. (default "cgrates")
-timezone string
Timezone for timestamps where not specified <""|UTC|Local|$IANA_TZ_DB>
-to_stordb
Import the tariff plan from files to storDb
-tpid string
The tariff plan ID from the database
-verbose
Enable detailed verbose logging output
-version
Prints the application version.

78
docs/cgr-migrator.rst Normal file
View File

@@ -0,0 +1,78 @@
.. _cgr-migrator:
2.4. cgr-migrator
-----------------
Command line migration tool.
Configurable via engine_configuration_ or command line arguments (with higher prio).
::
$ cgr-migrator -h
Usage of cgr-migrator:
-config_path string
Configuration directory path.
-datadb_host string
the DataDB host (default "127.0.0.1")
-datadb_name string
the name/number of the DataDB (default "10")
-datadb_passwd string
the DataDB password
-datadb_port string
the DataDB port (default "6379")
-datadb_type string
the type of the DataDB Database <*redis|*mongo> (default "redis")
-datadb_user string
the DataDB user (default "cgrates")
-dbdata_encoding string
the encoding used to store object Data in strings (default "msgpack")
-dry_run
parse loaded data for consistency and errors, without storing it
-exec string
fire up automatic migration <*set_versions|*cost_details|*accounts|*actions|*action_triggers|*action_plans|*shared_groups|*filters|*stordb|*datadb>
-out_datadb_host string
output DataDB host to connect to (default "*datadb")
-out_datadb_name string
output DataDB name/number (default "*datadb")
-out_datadb_passwd string
output DataDB password (default "*datadb")
-out_datadb_port string
output DataDB port (default "*datadb")
-out_datadb_type string
output DataDB type <*redis|*mongo> (default "*datadb")
-out_datadb_user string
output DataDB user (default "*datadb")
-out_dbdata_encoding string
the encoding used to store object Data in strings in move mode (default "*datadb")
-out_redis_sentinel string
the name of redis sentinel (default "*datadb")
-out_stordb_host string
output StorDB host (default "*stordb")
-out_stordb_name string
output StorDB name/number (default "*stordb")
-out_stordb_passwd string
output StorDB password (default "*stordb")
-out_stordb_port string
output StorDB port (default "*stordb")
-out_stordb_type string
output StorDB type for move mode <*mysql|*postgres|*mongo> (default "*stordb")
-out_stordb_user string
output StorDB user (default "*stordb")
-redis_sentinel string
the name of redis sentinel
-stordb_host string
the StorDB host (default "127.0.0.1")
-stordb_name string
the name/number of the StorDB (default "cgrates")
-stordb_passwd string
the StorDB password
-stordb_port string
the StorDB port (default "3306")
-stordb_type string
the type of the StorDB Database <*mysql|*postgres|*mongo> (default "mysql")
-stordb_user string
the StorDB user (default "cgrates")
-verbose
enable detailed verbose logging output
-version
prints the application version

58
docs/cgr-tester.rst Normal file
View File

@@ -0,0 +1,58 @@
.. _cgr-tester:
2.5. cgr-tester
---------------
Command line stress testing tool configurable via command line arguments.
::
$ cgr-tester -h
Usage of cgr-tester:
-category string
The Record category to test. (default "call")
-config_path string
Configuration directory path.
-cpuprofile string
write cpu profile to file
-datadb_host string
The DataDb host to connect to. (default "127.0.0.1")
-datadb_name string
The name/number of the DataDb to connect to. (default "10")
-datadb_pass string
The DataDb user's password.
-datadb_port string
The DataDb port to bind to. (default "6379")
-datadb_type string
The type of the DataDb database <redis> (default "redis")
-datadb_user string
The DataDb user to sign in as. (default "cgrates")
-dbdata_encoding string
The encoding used to store object data in strings. (default "msgpack")
-destination string
The destination to use in queries. (default "1002")
-file_path string
read requests from file with path
-json
Use JSON RPC
-memprofile string
write memory profile to this file
-parallel int
run n requests in parallel
-rater_address string
Rater address for remote tests. Empty for internal rater.
-redis_sentinel string
The name of redis sentinel
-req_separator string
separator for requests in file (default "\n\n")
-runs int
stress cycle number (default 100000)
-subject string
The rating subject to use in queries. (default "1001")
-tenant string
The type of record to use in queries. (default "cgrates.org")
-tor string
The type of record to use in queries. (default "*voice")
-usage string
The duration to use in call simulation. (default "1m")
-version
Prints the application version.

View File

@@ -6,7 +6,7 @@
Organized into configuration sections. All configuration options come with defaults and we have tried our best to choose the best ones for a minimum of efforts necessary when running.
Below is the default configuration file which comes hardcoded into `**cgr-engine**`_.
Below is the default configuration file which comes hardcoded into `cgr-engine`_.
.. literalinclude:: ../data/conf/cgrates/cgrates.json
:language: javascript

View File

@@ -41,7 +41,7 @@ master_doc = 'index'
# General information about the project.
project = u'CGRateS'
copyright = u'2012-2019, ITsysCOM'
copyright = u'2012-2020, ITsysCOM GmbH'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the