Report

INTRODUCTION:

This report presents the usability testing process for VirtuLearn, an Augmented Reality (AR)-based application specifically developed for final-year university students who are preparing for their internship programs. The application provides an interactive overlay of digital information onto real-world environments, allowing students to visualize and understand professional settings before beginning their actual internships. The target users of VirtuLearn are students with little to no prior exposure to the working world. By offering a contextual preview of workplace scenarios, the application aims to reduce anxiety, increase confidence, and improve student readiness before entering the job market.

The purpose of this usability test is to ensure that VirtuLearn is user-friendly, intuitive, and effective for its intended users. The test focuses on identifying usability issues related to navigation, interaction, content presentation, and overall user experience. Through this process, we aim to gather valuable feedback that can help improve the system’s design, functionality, and performance.

The main objectives of this usability test are to assess how easily users can interact with the AR environment, to determine whether the system effectively conveys relevant workplace information, and to evaluate user satisfaction during their experience. The report will also define several key terms that are central to the study, including usability, augmented reality (AR), user experience (UX), effectiveness, efficiency, and satisfaction.

Usability refers to the ease with which users can achieve their goals using the system, while augmented reality (AR) is a technology that enhances the real-world environment by overlaying digital content through devices such as smartphones or AR glasses. User experience (UX) describes the overall feelings and responses users have when using a product. Effectiveness relates to how accurately and completely users can complete tasks, efficiency measures the amount of effort or time required to complete them, and satisfaction reflects the comfort and positive attitude of users toward the application. By evaluating these aspects, this usability test aims to ensure that VirtuLearn delivers a high-quality learning experience for internship preparation.

 

METHODS & PROCEDURES

Participants

The usability test involved 4 university students from diverse academic. Participants ranged in age from 21 to 24 and were selected based on their prior experience with augmented reality (AR) technologies. None of the participants had previously interacted with the AR prototype, ensuring unbiased feedback. This sample was purposefully chosen to capture a variety of perspectives from students familiar with both the technical and non-technical aspects of the application.

Testing Tasks and Scenarios

Participants were instructed to complete four core tasks, designed to simulate realistic usage of the AR application intended to assist with internship preparation. The tasks were crafted to measure user experience, ease of use, and task success. These tasks were:

1. Exploring the AR Environment: Participants were asked to navigate a virtual company environment within the AR app. They needed to interact with different virtual objects and familiarize themselves with the layout of the space, simulating a realistic office tour for a potential internship.
2. Completing a Mission: In this task, participants were tasked with identifying and completing a mission within the AR environment. They needed to locate specific objects and complete interactive steps associated with the mission, such as gathering information or simulating actions related to the internship duties.
3. Accessing Skills Information: Participants were asked to search for information on required internship skills within the AR interface. This task tested how well users could navigate the app's content structure to find specific information relevant to the role they were simulating.
4. Providing Feedback: After completing the tasks, participants were required to submit feedback on their experience using a questionnaire. The feedback was designed to assess their satisfaction with the app’s usability, interface, and overall experience.

Testing Metrics

To ensure an objective assessment of the usability of the AR prototype, the following metrics were recorded:

• Task Success Rate: This metric measured the percentage of tasks successfully completed by each participant. Success was defined as completing the task without assistance or errors.
• Time on Task: The amount of time each participant spent completing each task was recorded to assess efficiency in the AR environment.
• User Errors: Errors such as miss clicks, incorrect navigation, or failure to complete a task were tracked. This metric provided insights into areas where the interface might be causing confusion or difficulty.
• User Satisfaction: A post-task questionnaire was administered to capture participant perceptions of the AR experience. Participants rated aspects like ease of use, clarity of task instructions, and overall enjoyment on a 5-point Likert scale (1 = Strongly Disagree, 5 = Strongly Agree).

Tools and Equipment

The following tools and equipment were utilized to conduct the usability test:

• Devices: The test was conducted using laptop, which is necessary for running AR . The devices were standardized to ensure consistency across all tests.
• Software: The AR application was designed using Figma for wireframes and mockups, integrated with an AR engine to render interactive AR environments and tasks.
• Recording: All testing sessions were recorded using phone camera, which allowed us to capture on-screen interactions for later analysis. Additionally, a video camera was used to record participants’ facial expressions and actions during testing for qualitative observations.

 

Overall Procedures

The usability testing was conducted in a controlled environment, ensuring consistency and minimizing external distractions. The overall testing process was as follows:

1. Pre-Test Briefing: Participants were briefed on the purpose of the usability test and given a brief introduction to the AR app. No specific instructions were given on how to complete the tasks to allow for natural exploration and interaction.
2. Task Execution: Participants completed the four tasks in sequence. They were encouraged to explore the AR app and complete the tasks to the best of their ability without external help. Each task was observed and timed, with the testers making note of any difficulties or errors.
3. Post-Test Debrief: After completing all tasks, participants were asked to provide feedback on their experience. They were given the questionnaire to rate their satisfaction and give verbal feedback. This debrief helped capture more qualitative insights into the usability of the app.
4. Data Collection and Analysis: The data collected from the task success rates, time on task, errors, and user feedback were analyzed to identify common issues or patterns. The results were used to make recommendations for improving the interface and functionality of the AR app.

RESULTS & ANALYSIS

User Task Completion Overview

User 1

Time completion: 5:22 minutes

Task	Success Rate	Average Time	Common Errors
Users need to be figuring out how to begin virtual Internship. Use available assistant to get guidance and follow any suggested steps.	100%	2:05 mins	Most Confusing – At Recommend for you
Users need to access about their performance after complete internship task. Check performance results and take note of feedback given.	100%	1:35 mins	-
Users find a way to reach out to a classmate. Start conversation and try to add them for contact.	100%	1:09 mins	Most Easy – Direct at menu

 

User 2

Time completion: 6:09 minutes

Task	Success Rate	Average Time	Common Errors
Users need to be figuring out how to begin virtual Internship. Use available assistant to get guidance and follow any suggested steps.	100%	2:36 mins	Most Confusing – Button and “Recommendation for you”
Users need to access about their performance after complete internship task. Check performance results and take note of feedback given.	100%	1:04 mins	Most Easy – Direct after complete intern task
Users find a way to reach out to a classmate. Start conversation and try to add them for contact.	100%	2:08 mins	-

 

User 3

Time completion: 4:41 minutes

Task	Success Rate	Average Time	Common Errors
Users need to be figuring out how to begin virtual Internship. Use available assistant to get guidance and follow any suggested steps.	100%	2:05 mins	Most Confusing – Button and” Recommendation for you”
Users need to access about their performance after complete internship task. Check performance results and take note of feedback given.	100%	1:10 mins	-
Users find a way to reach out to a classmate. Start conversation and try to add them for contact.	100%	1:15 mins	Most Easy – Easy to find

 

User 4

Time completion: 6:46 minutes

Task	Success Rate	Average Time	Common Errors
Users need to be figuring out how to begin virtual Internship. Use available assistant to get guidance and follow any suggested steps.	100%	1:15 mins	Most Easy - Direct
Users need to access about their performance after complete internship task. Check performance results and take note of feedback given.	100%	2:10 mins	-
Users find a way to reach out to a classmate. Start conversation and try to add them for contact.	100%	2:45 mins	Most Confusing – Button Position and Box Frame line

 

Observation

-        User 1 confusing at our Task 1 on “Recommend for you”, User 1 thought below of “Recommend for you” have video or button.

-        User 2 also had same problem and confuse at “Recommend for you”.

-        User 3 felt confused when she explores at our task 1 at “Recommend for you” and she thought same as User 1 and User 2.

-        User 4 felt confused on Task 3, He got confused on “Friend” button and Friend List Box Frame.

 

Quantitative Feedback

On a scale of 1(low) to 5(high) we make an average for user satisfaction feedback about our app prototype on usability testing:

·        Ease of Use: 4(Average)

·        Enjoyment: 4.5(Average)

·        Satisfaction: 4.2(Average)

 

Insights

• Our app experience was engaging, but for first-time users need clearer guidance.
• Button labelling and position to make users see and responsive need improvements.

• Positive feedback showed that users valued realism and interactivity, indicating potential for high engagement if minor friction points are resolved.
• Our app design approach needs more improvements and follows HCI 8 Golden Rule. 

IMPROVEMENTS/ACTION TAKEN:

Problem 1: Confused Homepage Design

Before:

Too much imagery and overlapping elements: Violates Rule 4: Design dialog to reach closure and Rule 6: Make easy reversal of action possible — the interface is cluttered, without apparent end or direction.

Doesn't have a clear CTA: Violates Rule 2: Make universal usability accessible and Rule 1: Make consistency an objective — users are uncertain where to begin.

Purpose of the site unclear: Violates Rule 7: Promote internal locus of control — users are confused and lacking control.

Login/Sign-up buttons not prominent: Violates Rule 3: Provide informative feedback — no instructions on next steps.

 After:

Clutter reduced by removing background distractions → adheres to Rule 8: Reduce short-term memory load.

Clear title/subtitle added to make platform clearer → aids Rule 1: Consistency and Rule 7: Internal locus of control.

Standout CTA buttons like "Get Started" → adheres to Rule 3: Feedback and Rule 5: Error prevention through guiding the user.

Better spacing and visual hierarchy → obeys Rule 2: Universal usability and Rule 4: Closure.

Outcome:

More clarity, more visual guidance, and faster comprehension by the user. Users know what they need to do next, with low cognitive load.

 

Problem 2: Dizzying Flow in Chat & Add Friends Features

Previously:

Flow from role pages to chat not obvious: Breaks Rule 1: Aim for consistency and Rule 4: Design dialogs to result in closure — users uncertain what to do next.

Icons not clearly labeled or intuitive: Breaks Rule 2: Aim for universal usability and Rule 8: Minimize memory load — users have to infer icon functions.

No feedback when making friends: Breaks Rule 3: Provide informative feedback — users cannot tell if action succeeded.

Following:

Tooltips & icon names included → supports Rule 2: Universal usability and Rule 8: Minimize memory load.

Straightforward and simple flow (add with confirmation or click to chat) → supports Rule 4: Closure and Rule 6: Easy reversal.

Feedback through confirmation like "Friend request sent!" or icon change → supports Rule 3: Feedback and Rule 5: Prevent errors directly.

Result:

Increased confidence in interactions. The users now know how to interact and communicate, with the reactions of the system confirming their actions.

 

CONCLUSION

The usability testing session gave us a clearer picture of how users interacted with our AR practicum app and what kind of experience they had. Overall, the participants responded positively to the idea and design of the app. They found the concept of exploring a virtual workplace fun, engaging, and useful, especially as a way to prepare for real internships or future jobs. Many users said the app felt modern and exciting, and they were curious to explore more job roles through the virtual simulation.

At the same time, the testing also helped us discover areas that needed to be improved. One of the main issues was the ‘homepage’ , which several users found confusing. Some were unsure where to click first or how to get started, and this made the app feel less welcoming at the beginning. Another part that caused confusion was the chat platform and the "Add Friends" feature. Although users were interested in chatting and connecting with others, they found it unclear how to use this function properly or where to find certain actions.

Based on this feedback, we made some key improvements to help users have a better experience. We redesigned the homepage layout to make it easier to understand, with clearer labels and better organisation so that users can quickly find what they need. We also updated the flow of the chat platform and made the add friend process more straightforward, so that users can interact smoothly without getting stuck or lost. These changes helped improve the overall usability of the app and made the navigation feel more natural.

This usability test was very helpful and showed us that even small changes can make a big difference in how users feel when using an app. It reminded us that designing a good product isn’t just about how it looks, but also about how people actually use it. We believe that with continued testing and updates, this AR/VR practicum app can become a valuable tool for students who want to explore real job environments in a virtual way. We are excited to keep improving it and hope it can support more students in preparing for their practical training in the future.