IoT fairs 2022: Smart testing for smart cities
QiTASC attended various trade fairs in 2022 to present our solution for IoT automated testing. We focused on the introduction of the portable testing lab in a suitcase for IoT. Wherever we went, it was the centre of attention. Next, we will attend the Smart City European World Congress from 15th to 17th November 2022 in Barcelona. Let’s meet up!
At fairs, stakeholders come together and exchange their knowledge about a particular topic. As an international company, QiTASC does not want to miss this chance, especially when it comes to one of our focused industries: IoT. We see that test automation benefits not just IoT but also telecommunications, finance, e-commerce and TICs. Although our focus has so far been on the telecommunications industry, we wanted to share our results and achievements in automated testing of IoT products.
IoT on its own is a broad topic. What we observed was that the many interest groups had one thing in common: They aim for a high-quality standard in their products and services. So far, so good, but the reality of the market does not make it easy. The time for developing a product is getting shorter. And when time runs short, you have to prioritise. As the last step in the development life cycle, it is often testing that is skipped in order to gain time for the product to hit the market.
QiTASC has a solution to this problem of short times for testing. And we wanted to present it on an international stage. We attended the IoT fairs to demonstrate our all-in-one test automation solution for smart cities that can cope with complex infrastructure and continuous integration. Our main goal is to ensure high quality in digital products even when time is short. We developed all the necessary tools for test automation, both software and hardware.
Presentation
IoT test automation with QiTASC
Gamze demonstrates the QiTASC way of using automated testing for IoT projects. You can download the slide deck used in the presentation.
shownotes
Testing IoT: The QiTASC way of automated testing of IoT projects
Hello, this is Gamze from the Sales Department. I will introduce our company to you and also introduce our smart system test automation.
QiTASC is the initial soft quality-improving tools and services company. We stress that our key task is providing and keeping high quality. We automate and validate digital processes and scenarios end to end. For this we develop the necessary tools.
We have developed intaQt, the intelligent test automation and quality assurance tool, and there are many possible areas of working with intaQt. We automate mobile devices, PC clients and MacOS, apps in tablets, apps in Android, trace integration, attenuators, VoIP phones, remote accesses, roaming, IoT devices, backend systems, web and app user interfaces, reporting, data correlation and verification, protocol and network simulation, and we also automate apps in iPhones. Therefore, we call intaQt the service pocket knife of automation.
Smart infrastructures need smart testing. This is why we do test automation. For companies who are developing digital products and services, continuous integration and continuous development is required nowadays. There is great demand for increased technical abilities from the customers which leads to more complicated systems to be tested.
On the other hand, time to market is tighter. This leads to a phenomenon called quality at speed. There is the need to increase the efficiency of testing. Therefore, test automation is essential.
Now I will talk about IoT test automation. There are two goals in IoT test automation. First of all, we have to ensure that the IoT devices work properly. Secondly, we have to ensure that they communicate with the apps properly. And there are challenges of course in IoT test automation. First of all, IoT devices automation requires physical interaction which is hard to automate. And secondly there is a multicontext framework which is hard to automate end to end within one scenario. We have IoT devices, apps, web user interfaces, backend systems, gateways, network elements and so on to be automated in one test scenario end to end.
How do we overcome these challenges? We developed a framework to automate IoT sensors. We developed housings in which we integrate actuators, sensors, motors, fans to enable physical interaction with the IoT devices. The actuators are put in housings that we design and develop to trigger the IoT devices. These actuators are connected to intaQt via an electronic layer.
This is the phase where we developed these housings. And then we put these housings on a platform and at the same time we can test hundreds of IoT devices, which is very efficient.
In this scheme we see the basic setup for testing 3 IoT devices. These actuators are connected to intaQt via a Raspberry Pi unit. These actuators trigger the IoT sensors and IoT sensors communicate with the apps in the smartphone via device and gateway domain and network domain. In this structure we automate the actors, we automate the app, and we also automatically collect all the data in the backend systems. Therefore, we automate the traces and log collection. A failure in this system can occur in various parts and there are hundreds of combinations of tests to be done which cannot be done manually. Automation is essential in this framework.
Once we set up this framework it is easier to repeat the tests. It is quicker to find the defects and it is quicker to go to the market. Therefore it is very convenient for the companies who are developing such smart systems.
This framework can be enhanced to various areas like wearables, smart cameras, smart meters, smart payment technologies, smart surveillance, mobile and desktop applications, broadcast streaming and signal quality, face recognition technologies, non-touch solutions, kiosks and various other areas.
There is a very nice example of our IoT test framework which is a small detector automation. You see a demo video here.
Our very innovative test automation frameworks which can be used in many different areas – this can be telecommunications, IoT systems, finance or health care systems – are preferred by lots of customers in various countries in Europe.
Some of the names of our customers are in this chart.
Thank you for listening. I hope you ask for a demo or if any questions arise, please contact us.
Tutorial Video
IoT test case using the software intaQt studio®
Bernhard explains an IoT automated test case step by step. The test case runs in the IoT setup of QiTASC´s test automation lab.
shownotes
Testing IoT: Step by step explanation of an IoT test case
Hello, my name is Bernhard from the electronics department of Qitasc. Today I would like to present an IoT project with an IoT test case.
The system we are testing consists of a system that is installed at home. A smart hub from the company Samsung in this case, and a door-window contact as an example. We can also use other devices, but in this case I would like to demonstrate a test case with a door-window contact. And a mobile phone.
We want to test the behaviour that you have at home. You have attached the door-window contact to a window and want to receive a notification on your mobile phone as soon as someone opens the window. Now we want to test both ends automatically. We do this by not mounting the door-window contact on a window, because how do you open a window automatically? We have mounted the door-window contact on a device that has a servo motor attached to it and this servo motor can open the window, so to speak. It folds the door-window contact away from the magnet. Here is the magnet, here is the door-window contact. This part can be folded away and closed again. The door-window contact is connected to the Samsung smart hub via the Zigbee radio protocol and the Samsung smart hub is connected to a backend server at Samsung via the internet. The phone, on which an app is installed, is also connected to the backend server at Samsung and via this connection, the connection to home is established. When the sensor detects that the window is opened, the information is transmitted to the backend server at Samsung. The phone is then notified that the window has been opened and is supposed to display this on the app. That’s what we want to test.
In order to do this, we have this servo motor. This servo motor is controlled by our electronics. This is the board with the electronics. Of course, it can do much more than just control the servo motor; all the functions are also integrated. It is important to know that there is a Raspberry Pi in the background. This Raspberry Pi is also connected to the network and can be controlled by us. It then controls the functions on this board. I can connect four devices to this board. In this case, only the first device is connected, this door-window contact. Via this platform that I have here, I can not only control a servo motor, I can also control many other functions. For example, I can also simulate the battery here. So, there is no battery inserted in this sensor, but a battery dummy is inserted. This battery dummy is supplied by this circuit board. With it I can also test, for example, the case when the battery becomes empty, that the device tells me that I have to replace the battery. I can also use it to switch relays and perform many other functions, adapted to the application which is needed for the automation of a smart home.
The test case is processed via intaQt studio as usual, which is our platform. Various steps are carried out in this test case, which are formulated almost naturally in terms of language.
We start by wanting to get the phone: Give my phone as A. Here even the serial number is given. That means I get a phone that has to be connected to my machine with this serial number. And insert a full battery into the device multipurpose sensor demo. The device here, this door-window contact, is a multipurpose sensor. It can do more than just door-window contact, that’s why it’s called a multipurpose sensor. And I put a full battery into that. Then I open the app on the phone. And close the window monitored by the sensor MultiPurposeSensor_Demo. I make sure that the servo motor brings the door-window contact into the initial position “closed window”. Then on A – that’s the phone, we got it and called it A – then on A, select the device MultiPurposeSensor_Demo from the devices menu. So on the phone the app is opened and in the menu of this app we go to the sensor we want to look at. And on A, assert that the contact sensor of the MultiPurposeSensor_Demo says “Closed”. We have previously set the sensor to the initial position “closed window” and now also want the app to indicate that the window is closed as an initial condition.
And then the actual test begins. Then open the window monitored by the sensor MultiPurposeSensor_Demo. This means that with this command, the servo will swing the sensor away from the magnet. This will look to the system as if the window is open. Then we give the system three seconds to wait. And wait for 3 seconds, so that there is really enough time for the sensor to notice this and for it to be transmitted. That’s often a problem with automated test cases, that you don’t expect the machine to be that fast. Much faster than you would be as a human being. The machine can do that almost simultaneously, but that would go wrong. So you have to give the sensor the opportunity to react, to transmit it all, to play through to the backend server at Samsung and then down to the phone. And that’s why there’s a three-second pause. It’s also easier for us to observe what’s happening that way, because it’s slower. And on A, assert that the contact sensor on the MultiPurposeSensor_Demo says “Open”. That means we want to see after three seconds that the app shows that it is open. And again we wait for three seconds. And wait for 3 seconds. Then close the window monitored. That means we shut down the sensor again. This time we even wait for six seconds and then we see that the app again shows that the sensor reports “Closed”.
I will now start the test case: The phone is now received from our test machine. The test case name is already displayed. Then the app is started. In the app, the sensor is navigated to. And then it is verified whether the initial condition is correct. It said “Closed”. It opened, now it says “Open”. After a waiting period, it then moves back to “Closed” and we also want to see that it says “Closed” again. This is the case. Test case is green and has the state “Passed”. So it worked.
We want to look at what would happen if the test case went wrong. Would we notice that? I will start the test case again and then I will intervene so that the window cannot be closed. The sensor will continue to report “Open” even though the window is supposed to close.
Now it has moved to the initial position “Closed”. It is navigated to the sensor. The sensor must show “Closed” as the initial condition. That is the case, it is “Closed”. Now it is opened. Now I hold my finger in between and make sure that the servo motor can no longer move the sensor back to the “Closed” position.
This means that it continues to display “Open” even though it has actually closed the window, and that is an error. And this error has also occurred here and has been detected, in this case: on A, assert that the contact sensor of the MultiPurposeSensor_Demo says “Closed”. But it still showed open. It is an error. It has been detected by our machine and displayed accordingly.
IoT test automation for smart cities
The IoT industry builds systems with devices that mainly use apps to control them. Apps have interfaces with digital elements that can be used in automated testing. Hence, the possibility of test automation in apps in obvious. On the other hand, the characteristics of IoT devices require physical interaction. Instead of pressing digital buttons or filling in forms in user interfaces, they have to be physically touched to be opened, closed, pressed, watched, etc. This is a challenge to automate.
IoT devices are placed in a multi-context framework. When attempting to test an IoT device, there is no obvious end-to-end scenario. When doing so, you have to focus on two components: On the one hand you have the software that executes the test cases. Its purpose is to navigate within apps or to collect data. On the other hand, you need some kind of hardware that simulates the physical interaction that is addressed by the software.
At QiTASC, we developed a test automation framework to automate the IoT sensors. Our aim is not only to offersoftware for test automation, but every component needed. And we want the tests to be end-to-end. With the intaQt® framework, each IoT test scenario can run in an automated way. We also offer complementary IoT devices such as hardware components, apps, web user interfaces, backend systems, gateways, network elements, etc.
To automate the physical interaction mentioned above, we work with actuators, sensors, motors, fans, or other small machines – depending on the test case. They are placed in small housings, which are designed in such a way that the actuators placed inside enable physical interaction with the IoT devices. The design of the housings has been developed and 3D-printed by our team to match requirements. The actuators inside the housings are connected to the intaQt® server via an electronic layer.
IoT testing lab to go
Usually, IoT test cases are executed in our testing lab consisting of actuators such as a door-window contact, a smoke detector or a smart light bulb. To give an example, the smart bulb can run tests on whether lights are turned on or off, if light intensity changes, what colour a bulb was, etc. Apart from the static solution, we offer an IoT test setup to go. The portable lab in a suitcase is a testing unit that can be carried or shipped anywhere and adjusted to a certain scenario. The suitcase is delivered wired and connected. Just plug it in and run your test cases.
We took the portable IoT testing lab to the fairs and visitors were amazed by its moving parts! The demonstrated suitcase was built to do smart home testing. It contains various actuators and can be used for different test cases for smart home testing.
Our observation of IoT fairs in 2022
In 2022, we have attended two trade fairs so far: The Mobile World Congress and the IoT Summit. Both the events took place in Barcelona, Spain. So will the next one, the Smart City European World Congress 2022. The topics of the fairs so far were mobile telephony, the internet of things and the smart city. Supported by the demonstrated IoT suitcase and its moving parts, the QiTASC sales team established contacts with exhibitors and fair visitors.
We noticed that the exhibitors were formed of two groups: the telecommunications giants were there, but the majority of the exhibitors were smaller companies. They offered technical niche products or special services. Usually, the target groups of those exhibitors are medium-sized companies which are looking for solution providers for the digital transformation of their products and services.
With regard to our customers so far, the benefit to telecommunications companies of our intaQt® framework is obvious. We have experience of millions of test cases in this industry. But it is not only the big ones that take advantage of our offer. Digital niche products in particular can benefit from automated testing. With our all-in-one solution they can offer high quality from the very beginning without wasting time on testing at the end of the product development. We would appreciate the opportunity to explain our part in any company`s digital transformation.
Conclusion
Good-quality implementation is secondary
At the end of 2022, we will have been at three fairs. Three times in Barcelona. Three times we presented our IoT suitcase. Our hardware solution that simulates physical interaction.
The fairs were an interesting opportunity to get an idea of the state of the art, not only in IoT. After the exchanges with many exhibitors and visitors, we drew our conclusions. The focus lies primarily on providing new telecommunications and IoT products. The market seems to put pressure on providers to present new technologies and services with a certain frequency. Good-quality implementation seems to be secondary. The drive to deliver is at the cost of the software that carries those products.
How do you see the trend? What was your experience at your last fair? What was your conclusion of the state of the art in IoT?
Let us know about your experience and send us a link to your website. And do not miss the opportunity to subscribe to our newsletter and fill out the questionnaire that helps you reflect on your business premiss for test automation.
