MTI uses practical guides such as How to Build a Speech Recognition Application and The Art and Business of Speech Recognition in the development of a speech application.
All speech application development is an iterative process and MTI's quotations for professional services in this area include allocations for continual refinement within the first twelve months of operation. Script, call flow, and grammars are evaluated for effectiveness. The use of utterance and whole-call recording, as well as analytic tools, allows for in-depth analysis of the application. Changes are expected and encouraged in the first year of operation with the goal of maximizing containment while providing an acceptable level of customer satisfaction.
Script development is usually a collaborative effort between the client and MTI. MTI can act as an advisor in producing a logical call flow in conjunction with the needs and desires of the client to insure a user-friendly design. Poorly designed voice response applications or overly elaborate web pages can be aggravating, thus producing a poorly executed application.
A user-friendly design:
• Provides clear product or service availability options, by location, when appropriate.
• Minimizes the number of inputs or responses from the user.
• Uses the appropriate style of menuing based on the complexity of user queries.
• Presents the information in segments, from lesser too greater detail (drill down).
• Allows the user to select the amount of detail they need.
• Provides an option for the IVR user to repeat segments or to skip ahead, on demand.
• Gives the user the ability to 'escape' to default information or an operator, as applicable.
A preliminary containment goal is always established based on client input and MTI experience.
If a speech application is being developed to augment or replace an existing live-agent application, then it is vitally important to secure the following items:
• Recorded examples of call interaction between the call center and callers
• All training materials used by the call center for agent training
• Meetings with call center agents to discuss "Best practices" not covered in written materials
Once a script has been finalized and has been approved by the client, grammars are proposed. Depending on the complexity of the script and grammars, early Wizard-of-Oz (WoZ) testing may be used to validate script flow and proposed grammars.
Scripts are then delivered to the programming team in order to develop call flow.
Prior to launch, both functional and usability testing is conducted. The following steps are employed in preparing for the testing:
• Test the test
• Create a pre-test questionnaire
• Develop a list of tasks to test
• Organize a logging method
• Identify test subjects
• Define the test environment
• Determine the analytics to be used in test interpretation
MTI will work closely with the Client to custom-design a quality assurance program specifically tailored to meet the program requirements. The program requirements will help to determine the number and types of evaluations performed each week.
During initial deployment, a multi-step evaluation process is used to examine VUI success in terms of usability. Areas such as product knowledge, sales, client service, data entry, etc., are monitored during this evaluation process. Automated exit interviews are encouraged as part of testing and evaluation.
1. Formal evaluations are conducted at various weekly intervals, dependent on Client requirements.
2. The use of informal monitoring is the method which QA uses to randomly 'pick up and listen' to calls.
A multi-phase deployment is preferred. This typically consists of three phases:
Pilot deployment - this consists of a few hundred calls and is an opportunity for programmers to tune the application. Tech nical glitches are addressed and design changes to prompts and grammars are made based on manual analysis of the call recordings.
Partial deployment - this step takes places only after all known issues have been addressed. Analysis typically takes place using automated tools, since the sample size is considerably larger than the pilot phase.
Full deployment - call volume is increased to full capacity and engineers and designers monitor the system
