Categorization profiles

A categorization profile contains categories and subcategories for the Conversation Analyzer feature. Conversation Analyzer uses the profile to categorize transcripts of call recordings. The profile also contains any substitution and redaction rules you provide. Using the substitution and redaction rules, Conversation Analyzer refines the transcribed text.

The categorization profile applies to the associated account. For information about where you can view the categorized recordings and refined transcripts, see Listening to and commenting on a call recording.

Categorization profiles are written in JavaScript Object Notation (JSON). For information about JSON, see https://www.json.org/.

Categorization profile structure

A categorization profile consists of the following top-level elements:

• name (a name/value pair)
• language (a name/value pair)
• categories (an array of category objects).
  Each category consists of the following:
  
  • name (a name/value pair)
  • rules (an array of one or more categorization rule objects).
    Each categorization rule object consists of the following:
    • party (a name/value pair)
    • expression (a name/value pair)
  • subcategories (an array of one or more subcategory objects).
    Each subcategory consists of the following:
    • name (a name/value pair)
    • rules (an array of one or more categorization rule objects).
      Each categorization rule object consists of the following:
      • party (a name/value pair)
      • expression (a name/value pair)
  For more information about categorization rules, see Categorization rules.
• substitution (an array of substitution rule objects).
  Each substitution rule consists of the following:
  
  • party (a name/value pair)
  • find (a name/value pair)
  • replace (a name/value pair)
  For more information about substitution rules, see Substitution and redaction rules.

name and language

Conversation Analyzer uses a call's Language and ConversationAnalyzerProfile data source values to identify the categorization profile to use to categorize and refine the call recording. Both language and name need to match the Language and ConversationAnalyzerProfile data source values to identify the profile. For information about how the Language and ConversationAnalyzerProfile data sources get their values, see Overview of Conversation Analyzer.

Categorization rules

As part of transcribing recordings, Conversation Analyzer categorizes the textual contents of the transcript, by identifying specific words and phrases that correspond to defined categories. A category is a collection of rules, with each rule consisting of a word or phrase and the party who said that word or phrase. If the transcript contains the word or phrase and was spoken by the specified party, the transcript matches the category.

For example, you may want to track how polite your agents are when speaking with customers. Create a category of 'Politeness' that looks for phrases such as 'Please', 'Thank you' and 'You're welcome'. You may also want to ensure that agents are promoting a new product or service. You would need to create a specific category that identifies incidences of the agent saying the product's or service's name.

Example categorization profile (category rules only)

In the following example, the categorization profile—Cat_example—contains one category—Cat details. Cat details contains two rules—one rule for each party—and one subcategory—Cat position. Cat position also contains two rules—one rule for each party. The substitution array contains no rules. If more than one rule applies to some text in the transcript, that text will appear in multiple categories.

{
    "name": "Cat_example",
	"language": "en-us",
    "categories": [
        {
            "name": "Cat details",
            "rules": [	
				{
                    "party": "customer",
                    "expression": "cat is ## years old"
                },
                {
                    "party": "agent",
                    "expression": "your cat?"
                }
			],
			"subcategories": [
				{
					"name": "Cat position",
                    "rules": [
                        {
                            "party": "customer",
                            "expression": "cat * sat mat ~2"
                        },
                        {
                            "party": "agent",
                            "expression": "cat mat ~3"
                        }
                    ],
                    "subcategories": []
                },
			]
		}
    ]
    "substitution": []
}

The following sections describe the party and expression name/value pairs.

party

party defines the party who must say the word or phrase defined by the rule expression for the transcript to match the category. Party can be customer or agent.

The format of party is "party": "value" where value can be:

• customer
• agent

expression

The expression name/value pair in a rule defines the text that must appear in the transcript to match the category.

The categorization expression language describes the format of an expression. The language supports simple expressions where the presence of the exact word or phrase would result in a match. For information about the categorization expression language, see Categorization expression language.

Substitution and redaction rules

Along with applying categorization rules to a conversation transcript, Conversation Analyzer applies substitution and redaction rules to refine the output:

• Substitution rules replace commonly mis-transcribed words and improve the spelling of words. You will most likely require these rules for proper nouns, such as place, company or product names. For example, Conversation Analyzer may transcribe 'Basingstoke' as 'Beijing spoke', or 'NewVoiceMedia' as 'new voice media'. Create rules that replace the incorrect word or words.
• Redaction rules replace sensitive information such as credit card details. Redaction rules are specific type of substitution rules in that instead of using them to refine and clarify phrases in the transcript output, you use them to obscure the content. Use a redaction rule to replace specified text with text such as '(redacted)', '(removed)', or 'xxxxxxxxxxxxxx'.

Example categorization profile (substitution and redaction rules only)

In the following example, the categorization profile—Subs_example—contains three substitution rules. The categories array contains no rules.

{
    "name": "Subs_example",
	"language": "en-us",
    "categories": [ ]
    "substitution": [
        {
            "party": "agent",
            "find": "new voice media",
            "replace": "NewVoiceMedia"
        },
        {
            "party": "customer",
            "find": "Beijing spoke",
            "replace": "Basingstoke"
        },
        {
            "party": "customer",
            "find": "my card number is *",
            "replace": "xxxx xxxx xxxx xxxx"
        }
    ]
}

The following sections describe the find and replace name/value pairs. For information about the party name/value pair, see party.

find

The find name/value pair in a rule defines the text that must appear in the transcript to match the substitution rule.

The categorization expression language describes the format of the value in the find name/value pair. The language supports simple values where the presence of the exact word or phrase would result in a match. For information about the categorization expression language, see Categorization expression language.

replace

The replace name/value pair in a rule defines the text that will replace the found text.

Applying substitution and redaction rules result in Conversation Analyzer modifying transcript text. Because of this, you must take extra care when writing your rules. For more information about substitution rules, see Substitution and redaction rules continued.

Categorization expression language

The categorization expression language describes the required format of the values you provide in the expression and find name/value pairs. Conversation Analyzer can then use these values to locate matching text in the transcripts.

Use the categorization expression language to define the categorization, substitution and redaction rules.

expression and find value validation

Valid expression and find values contain only alphanumeric, apostrophe and space characters; that is, values can contain spaces (U+0020), apostrophes (U+0027), and characters from the following Unicode categories:

Values can be no more than 100 characters long.

Wildcards in values

The categorization expression language supports the following wildcards within the values. Examples refer to the expression name/value pair, but exactly the same rules apply to find name/value pairs.

expression examples 

Example 1. Simple expression

"expression": "the cat sat"

With a simple expression, only the exact word or phrase will satisfy the rule.

Example 2. Expression using the ? character wildcard

"expression": "the cat? sat"

"expression": "sit*"

The * in the expression represents zero to many characters that can appear after "sit" in matching text.

Example 4. Expression using the # character wildcard

"expression": "### ###"

Matching text must contain two sets of three digits, separated by a non-word character and no other characters.

Example 5. Expression using the ?? wildcard

"expression": "wh?? cat"

"expression": "the cat sits * on the mat"

"expression": "cat mat ~3"

"expression": "cat sat mat ~3"

The text must contain the words "cat", "sat" and "mat" with up to three words between each of them. In this example, matching text may contain three words between "cat" and "sat" and also three words between "sat" and "mat".

Example 9. Expression using the ~N and * word wildcards

"expression": "cat * sat mat ~2"

Even when used with a ~N wildcard in an expression, a * word wildcard can represent any number of words. In this example, matching text can contain any number of words between "cat" and "sat", but a maximum of two words between "sat" and "mat".

Substitution and redaction rules continued

Overlapping substitution and redaction rules

Overlapping occurs when more than one rule matches the same transcript text. Because substitution and redaction rules actually modify the transcript text, overlapping rules can cause a conflict whereby multiple rules try to replace text with different values. To handle overlapping, Conversation Analyzer uses the following logic when applying the rules:

• The order of the rules in the profile determine their priority; the first rule has the highest priority.
• If rules overlap, the higher priority rule takes precedence over the lower priority. The lower priority rule is discarded.
• A discarded rule does not block any other lower priority rules.

Chaining substitution and redaction rules

Chaining occurs when one rule matches the output of another rule. Chaining only occurs when you re-analyze a recording. For information about re-analyzing recordings, see Configuring Conversation Analyzer.

Each time Conversation Analyzer applies substitution rules to a transcript, Conversation Analyzer overwrites the original transcript with the processed text. Rerunning the substitution rules can therefore further refine the text.

Highlighting replaced text

After Conversation Analyzer has processed a transcript, substituting or redacting text as your rules require, you are unable to see what has changed. If you want to see where in the transcript Conversation Analyzer, for example, removed text, create a category that highlights the replaced text.