How to configure transcription in Plugin Framework

The transcription feature allows transcribe conversion for customer voice/spoken dialogues in our Plugin Framework. Anywhere365 Dialogue Studio in combination with transcription, can be used to write the text to CRM Customer Relationship Management, or CRM, is (usually) a software-based, data management method to deal with interactions with customers and potential customers. or a database.

Requirements

• Core v 8.0 or higher
• SharePoint Template 6.0 or higher
• Google Cloud, with "Cloud Speech-to-Text API" enabled and json credentials

​​Transcription in Plugin Framework

There are a few settings required t​o allow plugins to use transcription functionalities. First two optional settings can be set in the general SharePoint UCC A Unified Contact Center, or UCC, is a queue of interactions (voice, email, IM, etc.) that are handled by Agents. Each UCC has its own settings, IVR menus and Agents. Agents can belong to one or several UCCs and can have multiple skills (competencies). An UCC can be visualized as a contact center “micro service”. Customers can utilize one UCC (e.g. a global helpdesk), a few UCC’s (e.g. for each department or regional office) or hundreds of UCC’s (e.g. for each bed at a hospital). They are interconnected and can all be managed from one central location. Setting list and allow transcribed text to be written to the general UCC log file for developer debug reasons. The other settings are plugin specific and go into the "Plugin Settings" list. See table below.

Settings

Plugin Settings

​​​Plugin specific settings

​​​​Google Speech

To setup Google Cloud follow this guide: https://cloud.google.com/speech-to-text/docs/quickstart-client-libraries​.

Make sure the "Cloud Speech-to-Text API" is enabled.

The following settings need to be added in the "pluginsettings" list.

Proxy server with Google Speech

To use a proxy server with Google Speech you will need to add an Environment Variable to your system (not to user only). Add the variable http_proxy​ with the value http://proxyserver.local:8080 .

Transcription Recording

Introduction

The transcriptions can also be recorded in a text file.

Settings

Location

The Recording will be stored at the same location as the audio recordings, Learn More

Data Format

In the example transcription below, you see that transcriptions have an "IsFinal" flag that indicates if the transcription has completed. Intermediate results are also logged. To determine the name or uri of the participant for which audio was transcribed, the TranscriptHistoryMessage.ParticipantId can be joined with the HistoryParticipant.Id.

{
   "Version":"1.0",
   "DialogueId":"f748e530-6955-47a6-b7da-0de1fe0d8ea3",
   "HistoryParticipants":[
      {
         "Id":0,
         "ParticipantUri":"sip:ucctestuser1406@anywhere365.net",
         "ParticipantDisplayName":"ucctestuser1406@anywhere365.net",
         "ParticipantType":"Customer"
      }
   ],
   "ChatHistoryMessages":[
      
   ],
   "TranscriptHistoryMessages":[
      {
         "Language":"nl",
         "Transcript":"Hello",
         "IsFinal":false,
         "ParticipantId":0,
         "Timestamp":"2020-01-06T11:38:34.8081659+01:00",
         "Index":0
      },
      {
         "Language":"nl",
         "Transcript":"Hello world",
         "IsFinal":true,
         "ParticipantId":0,
         "Timestamp":"2020-01-06T11:38:34.8236803+01:00",
         "Index":0
      }
   ],
   "TranslationHistoryMessages":[
      
   ]
}

Transcription in Core 6.0 or lower

Introduction

The transcription feature can writes the transcriptions directly to CRM.

Requirements

• A default configured UCC
• Azure Machine Learning Service
• (to write to CRM) Dynamics365 plugin v 6.0.19023

Configure

To configure this functionality the following SharePoint site settings are needed:

​Name	​Description
​UseTranscripting​​	​Enables or disables the functionality
​​TranscriptingServiceRegion	The Region to use for the Azure Transcription Service. If left empty Transcription will be disabled. Can be one of: eastasia southeastasia australiaeast northeurope eastus eastus2 southcentralus westcentralus westus westus2 brazilsouth
​TranscriptingLocale​​	Language used for transcription withe Azure Speech to text API.
​TranscriptingKey	​AzureKey for the Speech to text API.

​Transcription output

​CRM Plugins write Transcriptions to CRM.

All Transcriptions are written to the log and take the following format:

Copy

Format

Logger.Info($"TranscriptorRecognition: {leg.OutsideRealTimeAddress.Uri}. {e.TotalSampleDuration,10}, {e.SampleDuration,10}, {e.RecognitionStart,8}, {e.RecognitionDuration,8}. Phrase recognized: {confidence}. {phrase}")

or

Copy

Format

Logger.Info($"TranscriptorRecognition: {leg.OutsideRealTimeAddress.Uri}. {e.TotalSampleDuration,10}, {e.SampleDuration,10}, {e.RecognitionStart,8}, {e.RecognitionDuration,8}. Silence: {confidence}.")​

Configure Dynamics365 plugin

The Dynamics 365 plug in folder contains "Wsp.Anywhere365.Crm.Module.Dynamics365.dll.config", in here the plug in can be configured.

Change the following line:

Copy

XML

<add key="CreatePhoneCallActivity" value="true"/>

The plugin will now add the transcript to PhoneCall Activity note.