Service Monitoring Overview #

Located at the top-right corner of the dashboard, ‘Service Monitoring’ provides you with a visualization of the health of your channels by conducting tests with pre-defined parameters. 

Setting Up Service Monitoring #

To configure Service Monitoring using QuickTest, follow the steps below:

1. Create a Sprint Folder

• Navigate to the ‘Test Method’ section and select ‘Sprint Folders’.
• Click on ‘+ Create Folder’, enter a suitable folder name, and optionally select a SubOrganization Name.
• Once created, click the edit icon next to the folder to begin configuration.

2. Import Existing Sprint

• Click on ‘Import Sprint’ to add a sprint to the folder.

3. Configure Schedule

• Set the Date & Time for when the test should run.
• Select the appropriate Time Zone.

4. Set Pause Schedule (Optional)

• To temporarily pause testing during specific times, check the ‘Pause Schedule’ option.
• QuickTest will automatically resume testing after the pause period ends.

5. Configure Repetition Settings

• Click ‘Repeat’ to define how frequently the tests should run (e.g., every 5 minutes, hourly, daily, monthly).
• Specify the start and end dates, or create a custom repetition pattern as needed.

6. Notification Settings

• Email Notifications:
  Enter the email address(es) to receive test notifications. Use commas without spaces to separate multiple emails, e.g.,
  mail1@example.com,mail2@example.com
• SMS and WhatsApp Alerts:
  Enable ‘Send SMS’ and/or ‘Send WhatsApp Notification’. Enter phone numbers including country code, separated by commas with no spaces, e.g.,
  +448000000000,+448000000001
• Webhook Alerts:
  Enable ‘Send Webhook URL’ and provide a valid webhook URL to receive real-time test alerts in your preferred system.

7. Execution Settings

• Run Sequentially:
  Enable the ‘Run Sequentially’ option to ensure tests run one after another, with each test starting only after the previous one completes.
• Add Comments:
  Use the ‘Add Update Comments’ field to include notes or reminders related to the sprint configuration.

8. Handle Test Failures

• Re-run Failed Tests:
  Activate ‘Re-Run Failed’ to retry failed tests automatically. Set:
  • ReRun Count: Number of retry attempts (up to 5).
  • ReRun Minutes: Interval (1–5 minutes) between each retry.
  The test will retry until it passes or the retry limit is reached.
• Use Alternate Providers:
  Select ‘Rerun with Different Providers’ to automatically rerun failed tests using a secondary carrier. This helps diagnose whether the failure was carrier-related.

9. Edit Sprint Configuration

• Use ‘Edit Sprint Config’ to make quick changes such as updating the speech model or prompt timeout.

10. Individual Failure Alerts

• Enable ‘Send Email for Individual Failure’ to receive alerts for each failed sprint within the folder.

11. Save or Run

• Once configuration is complete, click ‘Save’ to retain your settings or ‘Run Now’ to execute the test immediately.

12. Enable Service Monitoring

Important: Ensure the ‘Use Service Monitoring’ checkbox is selected for this configuration to be included in your active service monitoring setup.

Defining Custom Parameters for Service Monitoring #

Before enabling Service Monitoring, you must define the evaluation parameters. Follow the steps below:

1. Click on your profile icon located in the top-right corner and select ‘Configuration’.

2. Click on the ‘Service Monitoring’ tab and then select ‘Service Monitoring Data’.

3. Enter a Name and select the Channel Name from the dropdown.

4. Choose the required Service Monitoring Parameters from the dropdown.

5. Define the following parameters by setting the minimum and maximum values that classify test results as Successful, Satisfactory, or Failed:

• Answering: Set min and max response time (in seconds).
• Correct Prompts: Set min and max accuracy range (in %).
• Welcome Prompt: Set min and max accuracy range (in %).
• Responsiveness: Set min and max response efficiency range (in %).

6. Click ‘Save’ to store the configuration.

Service Monitoring Overview #

1. Once you have created your Sprint Folder and defined the parameters, go back to your dashboard and click on ‘Service Monitoring’.

2. The folders you created in Sprint Folders will now appear in the Service Monitoring view.

3. Select the relevant test folder to view the quality of customer interactions across channels.

4. Performance is evaluated based on four parameters:

• Correct Prompt
• Answering
• Welcome Prompt
• Responsiveness

5. Each test run is represented by a colored dot:

• Green: Successful – met all defined parameters
• Yellow: Satisfactory – partially met parameters
• Red: Failed – did not meet the parameters

6. Hover over and click on any dot to view the test history, including the reason for test success or failure.

7. An Audio Quality graph is also generated. Hover over the dots on the graph to see the date, time, and MOS score.

Note: Audio quality data is available 60 minutes after test execution.