The Trigger Calibration Reports can be accessed via the Analytics dropdown menu. These reports are useful for modifying the fraud trigger thresholds based on historical call data. These reports are most useful after ClearIP’s fraud control has analyzed calls for at least a few weeks.
Generate Trigger Calibration Reports
The Trigger Calibration Reports enable better selection of fraud trigger threshold values based on historical call traffic; reports can be generated by entering details into the form based on the following information:
- Trigger Type - Each report is based on the Trigger Type. Select the Trigger Type based on which triggers are configured and for which data would be available. Depending on the Trigger Type selected, there will also be optional filters for viewing data for a specific Calling Number, Called Number, Service Provider, Group, and/or User. These filters can be left blank to not apply a filter.
- Weeks to Analyze - Each report looks at a minimum of 1 to a maximum of 12 past weeks of data. Select the number of Weeks to Analyze based on how much data would be available for the selected Trigger Type.
Once the form is filled out, click ‘Run Report’ to generate the report.
Analyze Trigger Calibration Reports
Depending on the Trigger Type of the report, there will be different viewable columns, and each report by type should be interpreted slightly differently. Below is the breakdown of the Trigger Calibration Report and how it should be interpreted.
As a reminder, each Trigger Type has a defined analysis period length and fraud score unit.
| Fraud Trigger Type | Time Period |
|---|---|
| Targeted Pumping | 15 minutes |
| Fast Traffic Pumping | 5 minutes |
| Slow Traffic Pumping | 60 minutes |
| Theft of Service | 60 minutes |
| Robocalling | 60 minutes |
| Wangiri | 5 minutes |
Data Displayed
ClearIP will only display the data of all calls for which fraud analysis is enabled. Data for calls that are not monitored for fraud will not be displayed in the report. For example, if fraud analysis is disabled on domestic calls, data for domestic calls will not be displayed in the report.
These reports will combine data for all calls to include data for both legitimate calls and calls suspected of being fraudulent. In the reports, calls where the maximum fraud scores exceed the fraud score threshold value are visible, as well as calls where the fraud scores fall below the fraud score threshold values. This allows the comparison of the data for legitimate traffic side by side with the data for potentially fraudulent traffic.
Maximum Fraud Score
The Maximum Fraud Score represents the maximum of the fraud score unit (for a particular trigger type) which was detected between the call source and destination in a single time period for that trigger type.
For example, on a Targeted Pumping by User report, the Maximum Fraud Score for any row in the report shows the maximum number of calls made between the User and a specific Called Number in a 15-minute period over the course of the Weeks to Analyze period. On a Fast Traffic Pumping by User and Calling Number report, the Maximum Fraud Score for any row in the report shows the maximum cumulative cost of calls made between the User and Calling Number pair and a specific Called Country in a 5-minute period over the course of the Weeks to Analyze period.
All reports are sorted by Maximum Fraud Score in descending order. This helps to show which call sources and destinations for a particular trigger type might be fraudulent for Weeks to Analyze period. It can also help in readjusting fraud thresholds to make them more accurate.
Period Count
The Period Count represents the number of periods where at least 1 call was made between the call source and destination (for a particular trigger type) in the time period for that trigger type.
For example, on a Targeted Pumping by User report, the Period Count for any row in the report shows the number of 15-minute periods in which at least one call was made between the User and a specific Called Number in the Weeks to Analyze period. On a Fast Traffic Pumping by User and Calling Number report, the Period Count for any row in the report shows the number of 5-minute periods in which at least one call was made by the User and Calling Number pair and a specific Called Country in the Weeks to Analyze period.
A high Period Count indicates that the call source regularly makes calls to the destination. This can be useful in adjusting triggers to help reduce false positives between specific call sources and destinations or to determine specific call sources and destinations for which to make new triggers to block potentially fraudulent traffic.
SIP Count
The SIP Count represents the total number of calls made between the call source and destination (for a particular trigger type) throughout the Weeks to Analyze period.
For example, on a Targeted Pumping by User report, the SIP Count for any row in the report shows the total number of calls made between the User and the specific Called Number over the Weeks to Analyze period. On a Fast Traffic Pumping by User and Calling Number report, the SIP Count for any row in the report shows the total number of calls made between the User and Calling Number pair and the specific Called Country over the Weeks to Analyze period
The Period Count and SIP Count data combined can help determine how the calls are spread over time. For example, a Period Count of 1 with SIP Count of 10 indicates that 10 calls were placed within the exact same period of 5 minutes in the case of Fast Traffic Pumping. However, a Period Count of 10 with SIP Count of 10 indicates that 1 call was placed in a 5-minute period for 10 different 5-minute periods. The former might indicate fraudulent activity while the latter might not, which can help with readjusting fraud score thresholds.
P-XX Fraud Score
The P-XX Fraud Score columns represent the XXth percentile fraud score between the call source and destination (for a particular trigger type) throughout the Weeks to Analyze period. The XX values are 99, 95, 90, 80, and 50.
For example, on a Targeted Pumping by User report, the P-99 Frauds Score shows the cumulative number of calls between the User and a specific Called Number for a 15-minute window that is greater than 99% of the fraud scores of all other 15-minute windows. In other words, for 99% of the periods analyzed, the call count between the User and the Called Number was the value in this column or less. Similarly, the P-95 Frauds Score shows that for 95% of the periods analyzed, the call count between the User and the Called Number was the value in this column or less.
On a Fast Traffic Pumping by User and Calling Number report, the P-99 Frauds Score shows the cumulative cost of calls between the User and Calling Number pair and a specific Called Country for a 5-minute window that is greater than 99% of the fraud scores of all other 5-minute windows. In other words, for 99% of the periods analyzed, the cumulative call cost between the User and Calling Number pair and a specific Called Country was the value in this column or less. Similarly, the P-95 Frauds Score shows that for 95% of the periods analyzed, the cumulative call cost between the User and Calling Number pair and a specific Called Country was the value in this column or less.
The differences between the various P-XX column values indicate how varied the fraud scores are. These columns can provide useful data for adjusting fraud score thresholds if the Period Count is not low. If the Period Count is very low, these columns may be ignored.
Compare fraud scores to current thresholds
The fraud scores in the Trigger Calibration Reports can be compared to the current fraud score threshold for each trigger type that is configured.
It is critical to scrutinize which calls are legitimate and which calls are fraudulent. If there are any fraudulent calls included in the report, the fraud scores for fraudulent calls tend to be drastically higher than the fraud scores for legitimate calls. This will help determine a clear threshold value that separates fraudulent and legitimate calls. If there are only non-fraudulent calls shown in the report, the fraud trigger thresholds should be modified to be high enough to prevent false positives, but low enough to quickly catch fraud.
Modify fraud score thresholds
Based on observations made from viewing Trigger Calibration Reports and comparing them to current fraud score thresholds, fraud score thresholds can be modified to better capture fraudulent activity and prevent false positives.
For example, below is some hypothetical data from a Fast Traffic Pumping by User report.
| Calling Number | Called Country | Maximum Fraud Score | P-99 Fraud Score | P-95 Fraud Score | P-90 Fraud Score | P-80 Fraud Score | P-50 Fraud Score | Period Count | SIP Count |
|---|---|---|---|---|---|---|---|---|---|
| 12345678900 | South Africa | 1.378 | 1.378 | 1.378 | 1.378 | 1.378 | 1.378 | 2 | 5 |
| 19876543210 | Jamaica | 1.165 | 1.165 | 1.025 | 1.025 | 0.582 | 0.326 | 21 | 35 |
| 14567891230 | Switzerland | 0.999 | 0.999 | 0.999 | 0.498 | 0.498 | 0.498 | 12 | 15 |
The above data shows that calling numbers tend to call South Arica, Jamaica and Switzerland with a fraud score of between 1 and 1.5, and assume they tend to call all other countries with a fraud score of 1 or less. Based on this observation, it can be decided that if any calling number calls to South Arica, Jamaica or Switzerland enough times within 5 minutes to accumulate a fraud score greater than 1.5, the calls are likely fraudulent. It can also be decided if any user calls any other country to accumulate a fraud score greater than 1 within 5 minutes, the calls are likely fraudulent.
To translate these decisions into fraud triggers, we can create the following triggers:
- A wildcard trigger that covers all calling numbers and all called countries with a default threshold value of 1.
- 3 exemption triggers that cover all calling numbers and South Arica, Jamaica, and Switzerland as the called countries with a default threshold value of 1.5.
These threshold values can be adjusted later if necessary.
Investigate call traffic
To see more details of call activity displayed in the Trigger Calibration Report, use the SIP Messages, SIP Reports, or SIP Charts pages and manually set specific filters.
For example, to see SIP messages relating to a specific Calling Number and specific Called Country, on the SIP Messages page under the Analytics dropdown menu, the Calling Number can be entered into the Translated Calling Number filter, and the desired country’s name placed in the Called Country filter.
These same filters can be applied on the SIP Messages, SIP Reports and SIP Charts as well.