At QiTASC, we've extended the scope of what we automate to areas previously left to manual testing. Between 2018 and early 2019, we've introduced and improved the following activities, which complement test creation, management and execution provided by intaQt:
- sQedule intelligent resource management and test case scheduling
- Device management and allocation with the intaQt Mobile App and intaQt Studio Phone Plugin
- The Device Abstraction Layer (DAL) allows any Android device to be configured and quickly integrated into the testing process
Audio Analysis and Comparison
- Confirming that the correct audio announcement is played
- Checking that the audio quality is within an acceptable range
- Speech-to-text & speech recognition automation
- Centralised project reporting and bug tracking
- Remote test case execution on phones and tablets
Additionally, 2019 will see the introduction of several huge productivity-enhancing products, which we'll be revealing in the next few months:
- disQover - A test case generation tool that automatically creates and updates entire suites of test cases using graphical models
- QiTASC PhoneHub - A modified USB hub that allows intaQt to control phone hardware including electrical currents, connected/disconnected states, and phone audio streaming capabilities via integrated soundcards
To solve this prioritization problem, we created sQedule, which is a resource-aware scheduler that schedules tests on Agents. sQedule evaluates and allocates phones, allowing for multiple test cases to be run in parallel by intaQt Agents, which requests test cases from the sQedule server. A list of all test cases are passed to the sQedule process, and sQedule will execute the maximum amount of possible test cases by optimizing the test cases properties and available phones. At QiTASC, we have run several thousand end-to-end regression tests within a day using sQedule.
This automated approach to device allocation ensures that test case scheduling is optimized according to available resources, and is especially useful when running regression tests or executing tests that are very long and must be run outside of employee working hours.
In addition to sQedule's automated resource management, the application also offers features for managing configurations and time constraints such as:
- Recognizing configuration switches or IoT devices
- Supporting time constraints, for example, to specify that certain test cases are only run on even days or during the last quart of an hour
- Test-repeat mechanisms, which repeat tests a certain number of times until passed
- Remote scheduling, allowing testers to schedule and assign tests from intaQt Studio even if they do not have a local intaQt
In all cases, sQedule executes tests as soon as possible, which helps manage devices and projects much more efficiently.
sQedule Product Page
sQedule Launch Article: How Resource-Aware Scheduling Improves Levels of Automation
The intaQt Phone Plugin, which is one of intaQt Studio's many productivity features, lets users interact with all devices in their project no matter where in the world they are. With a few clicks, users can check whether or not a phone is turned on and actually registered to a network, or close dialog boxes that unexpectedly show up.
Newsroom: Remote Apptest with intaQt
Documentation: UI Steps Language
Having access to live streams of devices is especially useful for identifying incorrect behaviors or figuring out of a step isn't working as intended. The intaQt Phone Plugin allows users to interact with devices on the screen in front of them in the rare cases where manual interventions might be needed.
When working with multiple team members who all have access to the same pool of phones, we observed several problems, such as:
- Not being able to see which users are accessing phones
- Not seeing if a phone is available/recognized by intaQt
- Not having an easy way of checking phone and subscriber characteristics
- Not hearing what audio is being played
In addition to livestreaming devices and allowing remote interaction, the intaQt Phone Plugin provides an overview of all phones in a project. Clicking on a phone reveals its properties, including hardware and network-related characteristics. It also shows if the phone is in use, and by whom, as well as the name of the test case the phone is assigned to.
Finally, it was time to give the QiTASC Autotest app a big update. The new intaQt Mobile App has a faster performance that is complemented by a cleaner appearance. The app now displays the tester and test that is currently using the phone, which simplifies tracking devices. The Mobile App is installed on each phone used by intaQt, which allows phones to be automated by QiTASC's software. The user interface gives users quick access to device information, and makes SIM card and MSISDN management much easier and more reliable. We’ve also optimized phone-to-server communications for the new app, meaning phone acquisition and property retrieval are now almost instant – phone properties are pushed to intaQt in real time.
Newsroom: Managing Phones with the New intaQt Mobile App
- Every Android model supports multiple Android versions
- Android versions may have slight implementation variations in the underlying code
- Even phones running the same Android version may have differences due the custom firmware that different manufacturers provide
For example, an end user might want to test the same phone model, such as the Samsung Galaxy S8, with all the different supported Android versions -- 7.0 (Nougat) to 9.0 (Pie) -- to ensure that the behavior of the device remains the same. Or, an end user wants to ensure that all devices running the latest Android version behave in the same way. However, Android OS segmentation and can threaten the reliability of testing or the ability to properly integrate a phone with test automation software.
To make testing multiple models as convenient and accurate as possible, QiTASC is introducing the Device Abstraction Layer or DAL, which allows any Android device to be configured and quickly integrated into the testing process. This feature enables the end user to take control of the phone integration process. With DAL, end users are no longer dependent on either the QiTASC phone integration process or the popularity of a certain device, meaning any required Android phone model can be configured with DAL and quickly integrated in the testing processes.
QiTASC wants to enrich and empower the ecosystem that it is surrounded by and to further the notion of open-source code and projects. For this exact reason, the DAL implementation of any device that already exists will be uploaded to a public repository anyone can access, improve or use DAL as a guide for integrating any device that is not already supported. Through the power of the community, it is expected that all popular devices will have a DAL implementation within a reasonable amount of time.
- Navigating IVR systems
- Listening to audio announcements
- Receiving network operator messages
Test cases that include audio attachments often require users to manually replay the recordings and check that the correct file was played. These manual efforts are time consuming and can greatly decrease the benefits of an otherwise automated test case. To solve this productivity issue, QiTASC developed Audio Fingerprinting, which is one of the features of our Audio Plugin. Audio Fingerprinting automatically compares the audio generated in a test case against a set of reference files and based on the files' "fingerprints" can determine if the correct audio was played. This is especially useful when testing long announcements and when announcements in foreign languages need to be checked.
Newsroom: Automated Announcement Verification with the intaQt Audio Plugin
Documentation: Match Audio
intaQt Speech Channel Monitoring is a robust feature that checks for predominant frequency, pauses and distortion. This type of testing is done by automatically playing audio reference files during the call, which represents sounds that might be transmitted between two phones during a call. The content and quality of these files are then compared against a set of requirements that define acceptable tolerance, pauses, frequencies, and acceptable deviations.
Newsroom: intaQt Audio Plugin
Documentation: Speech Channel Monitoring
intaQt Speech-to-Text functionality supports speech-to-text recognition of `.wav` audio files. This feature integrates services provided by the Google Cloud Platform Speech-to-Text API. The Speech-to-Text step specifies an audio file and its language and checks that it is converted to text correctly.
Documentation: Speech-to-Text Steps
QiTASC conQlude was created specifically to eliminate this problem by automatically uploading reports and test artifacts from all users in a project. Every test report is uploaded to a centralized location and parses the most important metrics such as test result, tester, defect numbers, duration and test case ID. Log artifacts that are automatically attached to the report include:
- `adb logcats` from participating phones
- intaQt Phone Service logs for participating phones
- Automatic screenshots of the phones when test cases fail
- The intaQt log
- All the custom attachments that the test creates
conQlude prevents data entry errors, inaccuracies and stops human errors from occurring. It also has proven to save test managers hours of time spent reviewing test data with an integrated approval workflow feature. For customers who use other tools like ticket tracking systems (such as YouTrack and Jira) or project management systems (such as HPQC), conQlude integrates seamlessly with these frameworks, and can both parse information from these systems and export test data to configured databases.
conQlude Product Page
Newsroom: Transparent Project Reporting with QiTASC conQlude
Documentation: conQlude Administration
We'll be gradually introducing the following, which you'll be able to read a lot more about soon:
QiTASC disQover is a model-based test case creation and management tool that lets users easily and instantly make changes to entire projects. Using graphical models and a templating framework, disQover helps the user quickly identify where a change needs to be made and adjust the necessary parameters. disQover then propagates the changes throughout the project in a matter of minutes. The application is use case and technology agnostic, meaning it is compatible with all the types of automated testing and devices that QiTASC supports, including:
- Telecommunications network testing
- UI Testing
- Backend and Protocol Testing
- Mobile phones, VoIP phones, desktop browser, tablets and virtual devices
In order to be as user-friendly as possible, disQover's complexity and mathematical logic stays "under the hood", while an intuitive graphical user interface guides users as they create and updated flow diagrams that represent the test project. disQover provides templates, which means models can be reused, tweaked for different projects or become the framework for any other similar projects. This means that as a project grows, or when new projects are being created, the set-up and maintenance time required to make all test cases accurate and up-to-date becomes shorter and shorter. This time saved from the initial process can now be allocated to valuable work such as fixing defects and focusing on project development.
disQover Launch Article - Introducing disQover - Model Your Test Scope and Discover How We Automated Test Case Maintenance
disQover Product Page
The QiTASC PhoneHub is a modified USB hub that provides better control and battery life to Android phones being tested by intaQt. These modifications reduce the amount of configuration required and limit the necessity to manually plug and unplug devices throughout a test project. The modifications allow intaQt to:
- Limit the electric current that can be consumed over USB by a phone
- Virtually unplug and re-plug a phone from USB
- Control audio streams via integrated soundcards:
- Devices connected to the PhoneHub can also be connected via audio cable
- intaQt can "listen" to audio coming from the phone and "talk" into the microphone to reflect a two-way phone conversation
- Audio streams for up to three phones can be used for test cases without any configuration required
This approach to managing the electric current significantly extends the battery life. In our experience, a battery will last only about one year if it is connected via USB at all times. However, its life can be extended to more than five years if the battery is in fact "used". Furthermore, by being able to virtually unplug and re-plug phones, testers no longer need to get up from their desks to adjust the phone. Even more importantly, if a remote phone loses connection, testers are no longer dependent on the availability of people at the remote phone's location, which is especially important when phones are located in a different time zone.
Users now have the ability to automate:
- Audio content analysis, comparison and quality assessment
- Test case scheduling and execution
- Device management and allocation for both test engineers and automated Agents
- Test reporting and defect tracking
- Speech-to-Text verification
Meanwhile, we'll be introducing tools that automate and manage:
- Test case creation using models that represent all of a project's test requirements
- Device integration and configuration, allowing all Android devices to be tested by intaQt
- Battery and soundcard usage
Best of all, these different features and products all integrate with each other seamlessly, which means that QiTASC customers will be able to manage, execute and analyze all of their testing activities from a single suite of tools. This reduces the redundancy that arises from using multiple test tools at different phases of the development cycle. We're looking forward to sharing more information about our new and forthcoming products and hope you are too!
Would you like to learn more about QiTASC's automation projects can benefit your testing activities? Contact us at firstname.lastname@example.org or visit our Contact page to speak to someone at QiTASC or arrange a demo!