Discuss project
By clicking the button, you consent to the processing of personal data and agree to the privacy policy.
SHOWCASE
How to prevent problems?
We have developed a bot testing system to detect problems before updates are launched
We have developed a bot testing system to detect problems before updates are launched
Automatic chatbot testing
AlfaStrakhovanie-Life is a specialized life insurance company of AlfaStrakhovanie Group. It is among the TOP 3 life insurers in Russia in terms of fees. AlfaStrakhovanie-Life works in three main areas: investment life insurance, life savings insurance, and credit life insurance.
Context
In the project for AlfaStrakhovanie-Life we have made a chatbot, which receives and processes data from several sources: databases and spreadsheets.
The chatbot communicates with several systems: webchat, Telegram, operator workstation, bot's knowledge base in Dialogflow.
In the project for AlfaStrakhovanie-Life we have made a chatbot, which receives and processes data from several sources: databases and spreadsheets.
The chatbot communicates with several systems: webchat, Telegram, operator workstation, bot's knowledge base in Dialogflow.
Problem
Systems on which the chatbot depends live and develop independently.
The system can be turned off for updates, maintenance or simply to become unavailable. The storage structure may change in the data source.
When an interaction failure occurs with the external system, we need to understand where the failure occurred and take action to restore functionality.
It could take several hours before the autotests were introduced from the appearance of the problem to the discovery and elimination of the cause.
Systems on which the chatbot depends live and develop independently.
The system can be turned off for updates, maintenance or simply to become unavailable. The storage structure may change in the data source.
When an interaction failure occurs with the external system, we need to understand where the failure occurred and take action to restore functionality.
It could take several hours before the autotests were introduced from the appearance of the problem to the discovery and elimination of the cause.
The task is to implement a monitoring system to track the performance of the chatbot, check the bot’s interaction with other systems and data sources.
Process
Cypress and Cucumber were chosen for automated bot testing.
Cypress is a modern framework for testing web interfaces. Cucumber is a tool for writing tests in the natural language of Gherkin. This combination allows you to separate the writing of test scripts by an analyst from the implementation of js-code testing by a developer.
Based on test scenarios, we can generate documentation on the system’s functionality. The documentation is always up-to-date and clear to the customer and any team member.
Cypress is a modern framework for testing web interfaces. Cucumber is a tool for writing tests in the natural language of Gherkin. This combination allows you to separate the writing of test scripts by an analyst from the implementation of js-code testing by a developer.
Based on test scenarios, we can generate documentation on the system’s functionality. The documentation is always up-to-date and clear to the customer and any team member.
End-to-end tests:
Verifying the interaction of two interactive systems: a webchat in the personal workplace and the operator’s workplace
The test simulates a user conversation with a client service specialist. The difficulty was to run two scripts on Cypress simultaneously within one test:
1. User access to a specialist through a webchat in a personal workplace.
2. Work of the specialist with the user through the web-interface of the workplace of the operator. Now Livetex service is used for AlfaStrakhovanie-Life project.
The scripts in the test were synchronized by running an additional server. In each script added commands that are sent to the server after actions in the system. For example, the user will not send a message to the bot until the server receives a message that the operator’s workstation has been authorized in the second scenario.
The test simulates a user conversation with a client service specialist. The difficulty was to run two scripts on Cypress simultaneously within one test:
1. User access to a specialist through a webchat in a personal workplace.
2. Work of the specialist with the user through the web-interface of the workplace of the operator. Now Livetex service is used for AlfaStrakhovanie-Life project.
The scripts in the test were synchronized by running an additional server. In each script added commands that are sent to the server after actions in the system. For example, the user will not send a message to the bot until the server receives a message that the operator’s workstation has been authorized in the second scenario.
Checking chatbot interaction with data sources: excel files and database
For the chatbot have developed special commands. Commands allow adding and deleting policies, clearing the current session, switching to the bot when communicating with the operator.
Got a session management system. With its help, you can set the context in which the chatbot will work.
A sample of policies is created to check the interaction of the bot with data sources. For each of these policies, different types of applications have been submitted, and applications are at different stages of consideration. In the test, policy numbers are added to the session, the bot requests data, and the result is checked against reference values.
For the chatbot have developed special commands. Commands allow adding and deleting policies, clearing the current session, switching to the bot when communicating with the operator.
Got a session management system. With its help, you can set the context in which the chatbot will work.
A sample of policies is created to check the interaction of the bot with data sources. For each of these policies, different types of applications have been submitted, and applications are at different stages of consideration. In the test, policy numbers are added to the session, the bot requests data, and the result is checked against reference values.
Functional tests:
Display and availability of webchat on AlfaStrakhovanie-Life personal workplace page
The test checks if there is a chat button on the page, auto-opens the chat window, and gets an answer to a simple question according to the knowledge base.
The test checks if there is a chat button on the page, auto-opens the chat window, and gets an answer to a simple question according to the knowledge base.
Functionality of the chatbot in Telegram
The test checks the availability of Telegram bot and the correctness of answers to questions about confidential data. When requesting protected data, e.g. data on contracts, the user will be recommended to go to the personal cabinet.
The test checks the availability of Telegram bot and the correctness of answers to questions about confidential data. When requesting protected data, e.g. data on contracts, the user will be recommended to go to the personal cabinet.
Results
Now we can regularly check the accuracy of the bot.
Running tests on a schedule allows us to quickly detect the problem and take action.
Auto-tests are built into the bot assembly conveyor. You can now detect problems before starting in the client environment.
Auto-tests work in Docker-containers. This makes it possible to build a flexible infrastructure for test management.
Order a chatbot
or get a consultation
or get a consultation