Top 8 automated testing use cases of 2019

Acceptance testing

Acceptance testing is comprised of Factory Acceptance and Field Acceptance test phases. This type of testing ensures that a system behaves correctly. It involves testing the software itself from an end user’s perspective. Acceptance testing is ongoing and may include:

• Checking release versions of external software implementations such as webdrivers
• Ensuring that newly-incorporated or updated code works
• Diagnosing and correcting defects
• Detecting regressions

Acceptance testing is especially important for ensuring full functionality when a product goes live, and to maintain high quality during ongoing product development and bug fixes.

Backend testing

Backend testing automates functionality on the server side of an application. This is important for confirming that interdependent or tightly-coupled components are communicating correctly as well as producing and storing data properly. Performing backend tests is critical for finding and fixing bugs early in a project’s development helps to ensure a robust infrastructure.
Backend testing typically involves testing and validating at least one of the following:

• APIs
• Databases
• Servers

Backend testing is becoming increasingly important as the complexity of integrated systems increases, and for maintaining regulatory compliance.

Tariff and charging verification

Tariff verification and charging conformity tests are critical for ensuring that convergent charging systems (CCS), online charging systems (OCS) and customer accounts are behaving correctly. These tests require accessing large amounts of metadata created by events such as phone calls or SMS transfers that affect charging, and involve verifying:

• Phone & subscriber details, such as phone type, phone number and service provider
• Call & event details, such as calling parties, lengths of time and passed/failed steps
• Internal infrastructure
• Legal requirements
• Financial characteristics
• Tariff configurations

Core network testing

Network operators follow system life cycles, involving regular system updates that, in turn, require system verification. Core network testing checks that the fundamental components of a network are behaving correctly over the course of a system life cycle. This includes regression tests for patches, updates, upgrades and network element replacement projects.

Examples of core network functionality that requiring testing include:

• Making and receiving different types of phones calls using:
  • Mobile phones (2G, 3G, 4G, WiFi, VoLTE)
  • VoIP phones
  • ISDN phones
  • POT phones
• Sending and receiving SMS
• Managing data connections and uploading/downloading content
• Roaming scenarios
• Verifying the speech channel of a call
• Verifying protocol messages of a call flow
• Verifying audio announcements

End-to-end testing

End-to-end testing checks real-world scenarios from start to end. This is done to ensure, for example, that the network behaves exactly the same as it would when triggered by a customer’s smartphone. These tests use real devices, meaning there is no need for injections to trigger core functionality. The advantage of using real devices instead of using a simulated input is that you can be confident in the validity of the device behavior and test outcomes.

End-to-end migration testing

Migration testing creates time-sensitive challenges, for example, when moving subscribers from an existing system to a new system. These tests must typically be performed within a very small time frame. For example, overnight or during a brief period of time in which few customers access a network’s functionality. By comparison, end-to-end tests of products that have not yet reached the market have more flexible timeframes.

Manually executing end-to-end migration tests means only a tiny subset can be executed before a decision must be made about whether to use the new system or return to the old one. Therefore, automating this type of testing is critical to ensuring a smooth migration.

IFRS testing

The International Financial Reporting Standards (IFRS) use a widely adopted, standard language for accounting and harmonizing financial statements, such as IFRS-9 and IFRS-15. These international standards are increasingly important as organizations establish business activities in multiple countries. Ensuring compliance to these standards is complicated and time consuming: Data centers typically require an interface that modifies their contents into IFRS-compliant structures.

IFRS verification happens offline, as opposed to in a live environment while transactions take place. This is because IFRS data is usually collected for analysis and verification after a certain amount of time has passed. By creating automated test cases, massive data structures can be checked against specifications quickly and reliably, which greatly reduces the time and resources dedicated to accounting activities.

Internet of things testing

The Internet of Things (IoT) involves connecting devices to the internet and controlling them via applications. Currently, IoT is most commonly associated with Smart Home devices, such as smart thermostats or alarm systems, where customers can access and control these devices using an app on their mobile device. The goal of IoT testing is to ensure that smart devices are both working properly and communicating correctly with their app.

Examples of Smart Home functionality include:

• Controlling a room’s thermostat over the Internet
• Having an alarm system that notifies the user about events
• Verifying that a smoke alarm detects particle density changes
• Controlling both optical and magnetic door/window contacts

Because many of these devices involve physical movement, such as a camera turning on/off or a window opening/closing, automated IoT testing is uniquely challenging.

Protocol testing

Protocol testing checks that data is transmitted between devices (such as routers, switches, and computers) over networks according to protocol specification. These protocol specifications define the rules that devices must follow when they communicate with each other over networks.

Depending on the types of network and data, protocols used may include:

• Session Initiation Protocol (SIP) – For example, when testing mobile phone calls over VoLTE or instant messaging
• Transaction Capabilities Application Part (TCAP) – For example, when testing mobile calls
• Diameter – For example, when testing data sessions such as downloading or uploading

These tests often capture network packets of an application on an interface and compare them against a published standard of the protocol. For example, during a phone a call, a test may start a trace file by executing an SSH script on a phone or Internet of Things (IoT) device. After the call ends, the SSH script is stopped and the trace file is downloaded and compared to see if it conforms to protocol. Because of the variety of formats and types, protocol testing often includes decoding messages before they can be checked.