Identifying Stale Index Records

In certain scenarios, the unique keys in index pods of the CDL are stale (presumed to be deleted). The NFs may try to use these unique keys for another record, as the CDL does not show the stale record details to the NFs.

The CDL allows the operator to enable a parameter to detect the stale records and perform the necessary action on such stale records.

To detect the stale records and perform actions:

  1. Identify the stale records. When a new record is created, the unique key is overwritten. To identify such records, enable the index-overwrite-detection parameter, and ensure that the unique-keys-prefix matches the prefix pattern.

  2. Perform the necessary action (notify, delete or log) on the identified stale records.

    The CDL detects any unique key that is overwritten and performs one of the following actions:

    • Deletes the stale record or the entire record. The delete action is triggered only if the PurgeOnEval flag is set to false for the stale record.

    • Notifies the NF about the stale record. The notify action sends the STALE_INDEX_NOTIFICATION to the NF.

    • Logs the overwritten unique key. For the log action, the stale record is logged with the WARN log level.

    Note

    If there are two different unique keys pointing to the same primary key; one with notify action and the other with delete action, then the notify action is performed.

Use the following configuration:

features index-overwrite-detection max-tps variable
features index-overwrite-detection queue-size variable
features index-overwrite-detection unique-keys-prefix uk
  action [delete-record, notify-record, log-record]
 

where,

  • max-tps: Controls the rate of notification for stale records. The default is 200.

  • queue-size: Controls the processing queue size of the stale records. The default is 1000.

    Note

    The queue-size parameter is for both delete and notify actions. The max-tps parameter is only for the notify action.

  • unique-keys-prefix: Specifies the unique-key prefix pattern along with the action that needs to be performed.

Example:

cdl datastore session
features index-overwrite-detection max-tps 250
 features index-overwrite-detection queue-size 2000
 features index-overwrite-detection unique-keys-prefix uk
 action notify-record
 exit    
  exit      
  

Troubleshooting

To troubleshoot the stale index records, set the index.overwrite.session logger to INFO level. The logs from the endpoint pods as well as the index pods help in troubleshooting.

CDL configuration:

cdl logging logger index.overwrite.session
 level info
exit
 

The following metrics are introduced:

  • overwritten_index_records_deleted–maintains the total number of records deleted due to stale records identified at index.

  • overwritten_index_records_skipped–maintains the total number of records detected as stale, but dropped when the queue is full while processing the records for notify or delete.