mirror of
https://github.com/cgrates/cgrates.git
synced 2026-02-11 18:16:24 +05:00
[Docs] Initial CDRe documentation
This commit is contained in:
DanB
@@ -280,7 +280,7 @@ const CGRATES_CFG_JSON = `
|
||||
"attempts": 1, // export attempts
|
||||
"field_separator": ",", // used field separator in some export formats, eg: *file_csv
|
||||
"attributes_context": "", // attributes context - empty disables attributes processing
|
||||
"fields": [ // template of the exported content fields
|
||||
"fields": [ // template of the exported content fields
|
||||
{"path": "*exp.CGRID", "type": "*composed", "value": "~*req.CGRID"},
|
||||
{"path": "*exp.RunID", "type": "*composed", "value": "~*req.RunID"},
|
||||
{"path": "*exp.ToR", "type": "*composed", "value": "~*req.ToR"},
|
||||
|
||||
248
docs/cdre.rst
248
docs/cdre.rst
@@ -1,5 +1,251 @@
|
||||
.. _AMQP: https://www.amqp.org/
|
||||
.. _SQS: https://aws.amazon.com/de/sqs/
|
||||
.. _S3: https://aws.amazon.com/de/s3/
|
||||
.. _Kafka: https://kafka.apache.org/
|
||||
|
||||
|
||||
.. _CDRe:
|
||||
|
||||
CDRe
|
||||
====
|
||||
|
||||
|
||||
TBD
|
||||
**CDRe** is an extension of :ref:`CDRs`, responsible for exporting the *CDR* events processed by :ref:`CDRs`. It is accessed via :ref:`CGRateS RPC APIs<remote-management>` and configured within *cdre* section inside :ref:`JSON configuration <configuration>`.
|
||||
|
||||
|
||||
Export types
|
||||
------------
|
||||
|
||||
There are two types of exports with common configuration but different data sources:
|
||||
|
||||
|
||||
Online exports
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
Are real-time exports, triggered by the CDR event processed by :ref:`CDRs`, and take these events as data source.
|
||||
|
||||
The *online exports* are enabled via *online_cdr_exports* :ref:`JSON configuration <configuration>` option within *cdrs*.
|
||||
|
||||
You can control the templates which are to be executed via the filters which are applied for each export template individually.
|
||||
|
||||
|
||||
Offline exports
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
Are exports which are triggered via :ref:`CGRateS RPC APIs<remote-management>` and they have as data source the CDRs stored within *StorDB*.
|
||||
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
|
||||
CDRe
|
||||
^^^^
|
||||
|
||||
**CDRe** it is configured within **cdre** section from :ref:`JSON configuration <configuration>`.
|
||||
|
||||
One **export profile** includes the following parameters:
|
||||
|
||||
export_format
|
||||
Specify the type of export which will run. Possible values are:
|
||||
|
||||
**\*file_csv**
|
||||
Exports into a comma separated file format.
|
||||
|
||||
**\*file_fwv**
|
||||
Exports into a fixed width file format.
|
||||
|
||||
**\*http_post**
|
||||
Will post the CDR to a HTTP server. The export content will be a HTTP form encoded representation of the :ref:`internal CDR object<CDR>`.
|
||||
|
||||
**\*http_json_cdr**
|
||||
Will post the CDR to a HTTP server. The export content will be a JSON serialized representation of the :ref:`internal CDR object<CDR>`.
|
||||
|
||||
**\*http_json_map**
|
||||
Will post the CDR to a HTTP server. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template.
|
||||
|
||||
**\*amqp_json_cdr**
|
||||
Will post the CDR to an AMQP_ queue. The export content will be a JSON serialized representation of the :ref:`internal CDR object<CDR>`. Uses AMQP_ protocol version 0.9.1.
|
||||
|
||||
**\*amqp_json_map**
|
||||
Will post the CDR to an AMQP_ queue. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template. Uses AMQP_ protocol version 1.0.
|
||||
|
||||
**\*amqpv1_json_map**
|
||||
Will post the CDR to an AMQP_ queue. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template. Uses AMQP_ protocol version 1.0.
|
||||
|
||||
**\*sqs_json_map**
|
||||
Will post the CDR to an `Amazon SQS queue <SQS>`_. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template.
|
||||
|
||||
**\*s3_json_map**
|
||||
Will post the CDR to `Amazon S3 storage <S3>`_. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template.
|
||||
|
||||
**\*kafka_json_map**
|
||||
Will post the CDR to an `Apache Kafka <Kafka>`_. The export content will be a JSON serialized hmap with fields defined within the *fields* section of the template.
|
||||
|
||||
export_path
|
||||
Specify the export path. It has special format depending of the export type.
|
||||
|
||||
**\*file_csv**, **\*file_fwv**
|
||||
Standard unix-like filesystem path.
|
||||
|
||||
**\*http_post**, **\*http_json_cdr**, **\*http_json_map**
|
||||
Full HTTP URL
|
||||
|
||||
**\*amqp_json_map**, **\*amqpv1_json_map**
|
||||
AMQP URL with extra parameters.
|
||||
|
||||
Sample: *amqp://guest:guest@localhost:5672/?queue_id=cgrates_cdrs&exchange=exchangename&exchange_type=fanout&routing_key=cgr_cdrs*
|
||||
|
||||
**\*sqs_json_map**
|
||||
SQS URL with extra parameters.
|
||||
|
||||
Sample: *http://sqs.eu-west-2.amazonaws.com/?aws_region=eu-west-2&aws_key=testkey&aws_secret=testsecret&queue_id=cgrates-cdrs*
|
||||
|
||||
**\*s3_json_map**
|
||||
S3 URL with extra parameters.
|
||||
|
||||
Sample: *http://s3.us-east-2.amazonaws.com/?aws_region=eu-west-2&aws_key=testkey&aws_secret=testsecret&queue_id=cgrates-cdrs*
|
||||
|
||||
**\*kafka_json_map**
|
||||
Kafka URL with extra parameters.
|
||||
|
||||
Sample: *localhost:9092?topic=cgrates_cdrs*
|
||||
|
||||
|
||||
filters
|
||||
List of filters to pass for the export profile to execute. For the dynamic content (prefixed with *~*) following special variables are available:
|
||||
|
||||
**\*req**
|
||||
The *CDR* event itself.
|
||||
|
||||
**\*ec**
|
||||
The *EventCost* object with subpaths for all of it's nested objects.
|
||||
|
||||
tenant
|
||||
Tenant owning the template. It will be used mostly to match inside :ref:`FilterS`.
|
||||
|
||||
synchronous
|
||||
Block further exports until this one finishes. In case of *false* the control will be given to the next export template as soon as this one was started.
|
||||
|
||||
attempts
|
||||
Number of attempts before giving up on the export and writing the failed request to file. The failed request will be written to *failed_posts_dir* defined in *general* section.
|
||||
|
||||
field_separator
|
||||
Field separator to be used in some export types (ie. *\*file_csv*).
|
||||
|
||||
attributes_context
|
||||
The context used when sending the CDR event to :ref:`AttributeS` for modifications. If empty, there will be no event sent to :ref:`AttributeS`.
|
||||
|
||||
fields
|
||||
List of fields for the exported event. Not affecting templates like *\*http_json_cdr* or *\*amqp_json_cdr* with fixed content.
|
||||
|
||||
|
||||
One **field template** will contain the following parameters:
|
||||
|
||||
path
|
||||
Path for the exported content. Possible prefixes here are:
|
||||
|
||||
*\*exp*
|
||||
Reference to the exported record.
|
||||
|
||||
*\*hdr*
|
||||
Reference to the header content. Available in case of **\*file_csv** and **\*file_fwv** export types.
|
||||
|
||||
*\*trl*
|
||||
Reference to the trailer content. Available in case of **\*file_csv** and **\*file_fwv** export types.
|
||||
|
||||
type
|
||||
The field type will give out the logic for generating the value. Values used depend on the type of prefix used in path.
|
||||
|
||||
For *\*exp*, following field types are implemented:
|
||||
|
||||
**\*variable**
|
||||
Writes out the variable value, overwriting previous one set.
|
||||
|
||||
**\*composed**
|
||||
Writes out the variable value, postpending to previous value set
|
||||
|
||||
**\*filler**
|
||||
Fills the values with a fixed lentgh string.
|
||||
|
||||
**\*constant**
|
||||
Writes out a constant
|
||||
|
||||
**\*datetime**
|
||||
Parses the value as datetime and reformats based on the *layout* attribute.
|
||||
|
||||
**\*combimed**
|
||||
Writes out a combined mediation considering events with the same *CGRID*.
|
||||
|
||||
**\*masked_destination**
|
||||
Masks the destination using *\** as suffix. Matches the destination field against the list defined via *mask_destinationd_id* field.
|
||||
|
||||
**\*http_post**
|
||||
Uses a HTTP server as datasource for the value exported.
|
||||
|
||||
For *\*hdr* and *\*trl*, following field types are possible:
|
||||
|
||||
**\*filler**
|
||||
Fills the values with a string.
|
||||
|
||||
**\*constant**
|
||||
Writes out a constant
|
||||
|
||||
**\*handler**
|
||||
Will obtain the content via a handler. This works in tandem with the attribute *handler_id*.
|
||||
|
||||
value
|
||||
The exported value. Works in tandem with *type* attribute. Possible prefixes for dynamic values:
|
||||
|
||||
**\*req**
|
||||
Data is taken from the current request coming from the *CDRs* component.
|
||||
|
||||
mandatory
|
||||
Makes sure that the field cannot have empty value (errors otherwise).
|
||||
|
||||
tag
|
||||
Used for debug purposes in logs.
|
||||
|
||||
width
|
||||
Used to control the formatting, enforcing the final value to a specific number of characters.
|
||||
|
||||
strip
|
||||
Used when the value is higher than *width* allows it, specifying the strip strategy. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Strip the suffix.
|
||||
|
||||
**\*xright**
|
||||
Strip the suffix, postpending one *x* character to mark the stripping.
|
||||
|
||||
**\*left**
|
||||
Strip the prefix.
|
||||
|
||||
**\*xleft**
|
||||
Strip the prefix, prepending one *x* character to mark the stripping.
|
||||
|
||||
padding
|
||||
Used to control the formatting. Applied when the data is smaller than the *width*. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Suffix with spaces.
|
||||
|
||||
**\*left**
|
||||
Prefix with spaces.
|
||||
|
||||
**\*zeroleft**
|
||||
Prefix with *0* chars.
|
||||
|
||||
mask_destinationd_id
|
||||
The destinations profile where we match the *masked_destinations*.
|
||||
|
||||
hander_id
|
||||
The identifier of the handler to be executed in case of *\*handler* *type*.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -283,7 +283,7 @@ asr_template
|
||||
templates
|
||||
Group fields based on their usability. Can be used in both processor templates as well as hardcoded within CGRateS functionality (ie *\*err* or *\*asr*). The IDs are unique, defining the same id in multiple configuration places/files will result into overwrite.
|
||||
|
||||
**\*err***
|
||||
**\*err**
|
||||
Is a hardcoded template used when *DiameterAgent* cannot parse the incoming message. Aside from logging the error via internal logger the message defined via *\*err* template will be sent out.
|
||||
|
||||
**\*asr**
|
||||
@@ -456,3 +456,59 @@ type
|
||||
**\*template**
|
||||
Specifies a template of fields to be injected here. Value should be one of the template ids defined.
|
||||
|
||||
value
|
||||
The captured value. Possible prefixes for dynamic values are:
|
||||
|
||||
**\*req**
|
||||
Take data from current request coming from diameter client.
|
||||
|
||||
**\*vars**
|
||||
Take data from internal container labeled *\*vars*. This is valid for the duration of the request.
|
||||
|
||||
**\*cgreq**
|
||||
Take data from the request being sent to :ref:`SessionS`. This is valid for one active request.
|
||||
|
||||
**\*cgrep**
|
||||
Take data from the reply coming from :ref:`SessionS`. This is valid for one active reply.
|
||||
|
||||
**\*diamreq**
|
||||
Take data from the diameter request being sent to the client (ie: *ASR*). This is valid for one active reply.
|
||||
|
||||
**\*rep**
|
||||
Take data from the diameter reply being sent to the client.
|
||||
|
||||
mandatory
|
||||
Makes sure that the field cannot have empty value (errors otherwise).
|
||||
|
||||
tag
|
||||
Used for debug purposes in logs.
|
||||
|
||||
width
|
||||
Used to control the formatting, enforcing the final value to a specific number of characters.
|
||||
|
||||
strip
|
||||
Used when the value is higher than *width* allows it, specifying the strip strategy. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Strip the suffix.
|
||||
|
||||
**\*xright**
|
||||
Strip the suffix, postpending one *x* character to mark the stripping.
|
||||
|
||||
**\*left**
|
||||
Strip the prefix.
|
||||
|
||||
**\*xleft**
|
||||
Strip the prefix, prepending one *x* character to mark the stripping.
|
||||
|
||||
padding
|
||||
Used to control the formatting. Applied when the data is smaller than the *width*. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Suffix with spaces.
|
||||
|
||||
**\*left**
|
||||
Prefix with spaces.
|
||||
|
||||
**\*zeroleft**
|
||||
Prefix with *0* chars.
|
||||
|
||||
55
docs/ers.rst
55
docs/ers.rst
@@ -178,6 +178,9 @@ filters
|
||||
**\*vars**
|
||||
Request related shared variables between processors, populated especially by core functions. The data put inthere is not automatically transfered into requests sent to CGRateS, unless instructed inside templates.
|
||||
|
||||
**\*tmp**
|
||||
Temporary container to be used when exchanging information between fields.
|
||||
|
||||
**\*req**
|
||||
Request read from the source. In case of file content without field name, the index will be passed instead of field source path.
|
||||
|
||||
@@ -293,6 +296,58 @@ type
|
||||
Specifies a template of fields to be injected here. Value should be one of the template ids defined.
|
||||
|
||||
|
||||
value
|
||||
The captured value. Possible prefixes for dynamic values are:
|
||||
|
||||
**\*req**
|
||||
Take data from current request coming from the reader.
|
||||
|
||||
**\*vars**
|
||||
Take data from internal container labeled *\*vars*. This is valid for the duration of the request.
|
||||
|
||||
**\*cgreq**
|
||||
Take data from the request being sent to :ref:`SessionS`. This is valid for one active request.
|
||||
|
||||
**\*cgrep**
|
||||
Take data from the reply coming from :ref:`SessionS`. This is valid for one active reply.
|
||||
|
||||
mandatory
|
||||
Makes sure that the field cannot have empty value (errors otherwise).
|
||||
|
||||
tag
|
||||
Used for debug purposes in logs.
|
||||
|
||||
width
|
||||
Used to control the formatting, enforcing the final value to a specific number of characters.
|
||||
|
||||
strip
|
||||
Used when the value is higher than *width* allows it, specifying the strip strategy. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Strip the suffix.
|
||||
|
||||
**\*xright**
|
||||
Strip the suffix, postpending one *x* character to mark the stripping.
|
||||
|
||||
**\*left**
|
||||
Strip the prefix.
|
||||
|
||||
**\*xleft**
|
||||
Strip the prefix, prepending one *x* character to mark the stripping.
|
||||
|
||||
padding
|
||||
Used to control the formatting. Applied when the data is smaller than the *width*. Possible values are:
|
||||
|
||||
**\*right**
|
||||
Suffix with spaces.
|
||||
|
||||
**\*left**
|
||||
Prefix with spaces.
|
||||
|
||||
**\*zeroleft**
|
||||
Prefix with *0* chars.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user