Virtual Reality
Virtual Reality (VR) is an immersive media experience replicating either a real or imagined environment and allowing users to interact with the environment. Unlike the conventional user interface of displaying content on a screen, VR immerses a person to interact in a digital 3D environment.
VR in Different Industries
Virtual Reality is getting its share across every industry. Below are few key areas where VR is being used:
- Healthcare – Virtual Reality is showing promise in providing solutions to many of the complex requirements in healthcare. Some of the use cases include Patient Education, Surgical Procedure Training, Physiotherapy & Rehabilitation, Stress Relief, Fitness, Doctor Visits and many more.
- Architecture – VR possibilities in architecture are second to none. There are numerous options to adopt VR in architecture like rendering architectural models, urban planning, landscaping, mapping etc.
- Tourism – VR is an excellent marketing tool that can be successfully used in tourism. Going forward it is not going to be sufficient to reach out to customers with just photos, videos, or texts. There is a huge potential in using VR for tourism in terms of exploring places, making booking decisions, recreating past experiences and many more.
- Engineering – The possibility to support early engineering design phases by means of VR, companies are able to save themselves from potential failures after the product launch. This includes virtual assemblies and even taking decisions based on prototypes. This reduces the risk of an engineered product failing after a full-fledged development.
- Retail – The Ecommerce experience of end users is all set to evolve with body scanning technology for identifying the right fits, virtual try-ons, product customizations etc.
And it does not stop here, the use cases expand to various other industries such as Defense & Military Services, Gaming & Entertainment, Education, Recruitment and much more.
Challenges in VR testing
- Continuous usage of Virtual reality equipment for testing creates health risks to the users including eye strain and motion sickness. It is highly recommended for testers to be aware of the safety standards to be followed on testing an application with a virtual reality equipment.
- With countless types of VR devices available in the market, the application can exhibit different behaviors in different types of devices. So, it is frequently required for testers to have access to actual devices or a lab with a variety of devices if they want to adequately test the applications.
- Approaching VR testing based on traditional testing practices and tools will not help in effectively validating the VR applications and its behaviors.
- Manually testing the applications is time and resource consuming and the test automation tools do not have extensive support for virtual reality testing.
- Applications for virtual reality use a lot of processors, graphical resources and memory which might cause sluggishness. So, testing the performance of such applications pose a big challenge.
- There are currently very few emulators available to test VR behaviors.
VR Testing Approach
Testing a VR application requires a different set of considerations and approaches compared to testing traditional, 2D applications.
1) Defining the Scope
It is very important to define the scope of testing at the start and so the team is well prepared for what is to be expected. Quality engineers should organize brainstorming sessions to sit down and review the requirements in scope of testing and their complexities.
Consider the various functional and non-functional characteristics of the application to be developed and define a test strategy. This strategy should act as a general guideline for the QA engineers.
Business Analysts and Quality Engineers should work closely to evaluate the requirements. Non-functional requirements related to software performance, accessibility, and the user experience should also be documented to avoid ambiguities during the implementation phase.
Shift left testing approach does not stop just in conventional testing, it is highly important for Quality Engineers to review UI/UX and provide early feedback of the application to prevent design flaws. If the test application can run on a desktop or laptop in 2D mode, schedule your testing early with the 2D mode. This will help reduce fatigue for testers who must operate the equipment in 3D mode.
2) Finalize the Devices
One of the critical points for testing a VR application is that you must have a physical environment to test in. Invest in testing equipment that can help validate functional and non-functional behavior. Devices that will possibly be used by end users should be considered while shortlisting the devices. Careful selection of test devices helps to ensure the application is tested at its intended way.
3) Collaborate with the Developers
Work closely with the developers to ensure adequate coverage in unit testing. Application development platforms, provide capabilities for automating unit tests and even measuring the coverage. It is important to have high unit test coverage that will help identify issues at the initial development phase and save the cost of getting a bug introduced in subsequent systems.
4) Perform Various Types of Testing
Functional Testing – Plan for Functional Testing based on business requirements. All the functionality of your VR software’s core functions should be tested to ensure consistent performance of all aspects of the application’s features. Think of all actions which a user would perform in a real device and ensure all possible edge case scenarios are covered in testing.
Localization Testing – Localization Testing should be done to ensure the content and specific functionalities developed for different countries are working fine in the VR application. Cultural elements like colors, symbols, etc. should provide a local user experience.
Usability Testing – Usability Testing is performed to check the level of ease in using the VR application. This ensures the use cases are aligned with the preferences and needs of the target audience and the UI/UX exhibit user friendly behavior. Feedback on usability issues should be documented for further improvements. This helps to assess the level of user satisfaction on their interactions with the application.
Immersive Testing – An Immersive Test gauges how much a user is immersed in a virtual environment. The test findings often demonstrate how fully immersed participants are in the application and how strong these sensations are.
Accessibility Testing – With people across the globe trying to access the VR application, for performing different tasks, it is important that the large segment of our society containing more than a billion people with certain disabilities (visual & hearing impairment, cognitive disability, motor impairments, etc.) can access the application the same way as any other person would. People with any disabilities such as hearing, or vision issues should be able to experience VR games with a very minimal discomfort.
Compatibility Testing – Compatibility Testing should be performed to ensure the VR application is compatible with different hardware and platforms that it is going to be used by the end users. Application should not contain a feature that would not work in certain VR devices or models. It is very important to test if the VR application is both forward and backward compatible.
Performance Testing – Performance Testing is performed to evaluate the VR application’s performance under load and stress conditions. This is highly relevant. Performance monitoring tools such as OVR metrics and Unity profiler should be used to collect performance related data of the application. Performance testing should be planned considering each platform in mind to ensure the application works well across all the platforms.
Security Testing – The need to identify vulnerabilities, threats, and risks in the VR application and protecting it from malicious intruder attacks is necessary to avoid loss of data, revenue and, most importantly, reputation. All potential loopholes and weaknesses in the application should be unearthed during the testing.
Automate (If possible)
Though there are no test automation tools/frameworks specifically designed to test all VR features, there are a few tools like AirtestIDE and Poco which can be leveraged to automate the VR tests and reduce some manual testing efforts.
Tools for Testing VR
Below are some of the tools that can be handy during testing:
- Unit testing framework helps to automate unit tests to identify issues at the earliest stages.
- AirtestIDE and Poco are image recognition-based testing platforms that can be used to automate regression scenarios.
- OVR Metrics Tool helps with performance monitoring data through an onscreen graph over a running app or a report that can be retrieved from the device. It also offers remote device management functionality.
- SteamVR helps to test the VR content in the hardware of your choice. It supports various devices such as Oculas and Windows MR
- Unity Profiler collects and shows data on the performance of your application in areas such as the CPU, memory, renderer, and audio. It’s a relevant tool to pinpoint areas for performance optimization in your application and run through those areas repeatedly.
Conclusion
To conclude, testing a VR application requires a comprehensive approach considering a wide range of factors. This approach ensures the QA Engineers deliver a high-quality and immersive user experience for VR applications.
