Since it began IVR hosting in 1991, the company has developed dozens of applications employing IVR and VUIs. We have developed testing methodologies which ensure that speech applications meet their program requirements.
Testing is divided into three categories:
Functional testing is handled in cooperation with the development team. Skilled technical personnel develop a test plan that is designed to exercise all elements of an application. That plan is executed by internal personnel. While all elements are tested, all permutations and combinations are not. However, all reasonable combinations, based on an understanding of normal call flow, are exercised.
Usability testing is both part of early application development and extends through the refinement of an application that takes place in the first year of operation. Early in application development, this includes WoZ testing, and grammar testing. Prior to deployment, MTI live agents as well as "friends and family" are used to test usability. Typically they are given a goal to accomplish, such as "Request a refund," or "Change your billing address." They are provided the necessary account information and the number to call. Depending on the scope and life expectancy of the project, groups may be invited to a central facility where their call attempts can be videotaped and monitored. More typically, friends and family will make these calls when convenient to them, from the comfort of their own homes. Care is exercised not to re-use individuals too often or for applications that are too similar in design.
During and after deployment, usability testing includes whole call monitoring, and speech analytics. Depending on the scope and life expectancy of the project, focus groups may be employed in the usability design.
Stress and load testing are segmented to include testing of the voice gateways, testing of the application servers, testing of the database servers and overall testing of the application. The extent of testing is dependent on the nature of the application. For example, an application where call volume is expected to be light (perhaps a few hundred calls per day) is load and stress tested less extensively than applications generating thousands of calls per hour.
-
MTI relies upon vendor testing of the voice gateways. Each vendor MTI selects has engaged in extensive load and stress testing of their products.
-
If MTI is engaged in providing a turnkey solution, MTI uses simulation software to impose a load on its application servers (and indirectly the database servers). The simulation software measures the time between a fetch request and the deliverable. It also checks to insure the deliverable is as expected. From the application server's perspective, this in effect is the equivalent of a telephony platform making fetch demands on the server. Thousands of requests can be simulated in this way.
-
If MTI is engaged in providing a turnkey solution, then upon completion of step 2, the company develops automated scripts to be followed to exercise the application point-to-point. These scripts are deployed on an automated platform that calls the automated platforms running the target application.
