Testing and, more specifically, test automation is the backbone of software product development. Developers need to be confident that changes they make are proven to work, while simultaneously not breaking any existing functionality. This quality assurance needs to be completed before handing the end result to the customer. Although part of this is achieved with manual testing, test automation has become an increasingly valuable tool to ensure consistent results.

Customized solutions a challenge for test automation

While real estate agents emphasize the importance of location in buying properties, we in the software business often emphasize the importance of test automation as part of quality assurance. Test automation ensures the functionality works as intended and with the required performance. Testing is a critical part of both managing risks and ensuring customer satisfaction. Testing ensures that any issues are identified and resolved in a timely manner. In test automation, but especially with concepts like Continuous Integration and DevOps, this is referred to “shortening the feedback loop”. It is well known in software development that the later an issue is found, the more costly it is to fix. This is one of the main challenges that test automation strives to meet.

Customized solutions, integrated to a customer’s IT infrastructure, have long been at the core of what VividWorks does, but they also present challenges for testing. A VividWorks solution may be provided as part of web shop, standalone AR application or a sales demo of, for example, a paper machine for a sales presentation or expo. Due to the fact that VividWorks has both a broad offering and over 100 customer environments, of which many are highly customized, we have a lot to test. This also sets the bar high for testing.

While a customized solution is well justified in many cases to meet customer needs, should be noted that there are downsides to this approach. From the perspective of VividPlatform™, it might mean that not all of the new features or even bug fixes can be applied to the customized solution without extra effort, potentially resulting in increased cost of maintenance. This might mean that test cases developed as part of the platform needs to be either discarded or modified to fit the custom use case.

One area that has proven especially challenging for test automation is that of testing of 3D elements in our application. Part of the challenge stems from the history of 3D technologies, the development of which has long been driven by the gaming industry. Historically, games have been developed in one big push, released, and then more or less forgotten in terms of maintenance or new feature development. This has resulted in the stance that test automation development might not always be the effort. It can also be said that it has resulted in a lack of tooling around testing and some related software development techniques. Test automation has been more emphasized in recent years including in the gaming industry, surely due in part to online gaming, where, the games are more long lived.

High Quality Rendering demands high quality testing

In order to promote products and services in the best possible way, VividWorks provides high quality rendered, nearly photorealistic images. High Quality Rendering or HQR can, for example, generate a photorealistic view of a living room for testing different interior designs.

HQR is only as good as the quality it provides, so testing is a crucial – and also demanding - part of this service. One functional path for testing HQR is to do integration testing with photo comparison. This method enables testing different parts of the process as one. As testing of the service was done from the API level, it was simple to utilize as part of performance testing as well. In this case, it allowed us to find some of the timing issues in 3D rendering that would have been hard to spot with regular testing or without image comparison.

There are many tools and solutions for testing and test automation, but as typical in the IT business, development speed is rapid and every version is supposed be better than the previous one. Here at VividWorks, testing is continuously sparring with product development. Testing goes hand in hand with the quality and quantity of technological solutions. With this in mind, we are paying close attention to the testability of our software.