Embedding Stealth Assessment in Metaverse-Based Training for PC Building: Advancing Technical Education Using Evidence-Centered Design

Introduction

Technical education is a cornerstone of modern life and work in a technology-powered world. As the global workforce increasingly relies on technological tools and digital platforms, understanding and working with technology has become a basic competency across industries (Beer & Mulder, 2020; Li, 2024). A systematic literature review highlights technical skills as one of three essential employability skill types, along with cognitive and non-cognitive skills, critical for Industry 4.0 and beyond (Fajaryati et al., 2020). Supporting this viewpoint is the recent report from the World Economic Forum (2025), which underscores that technical proficiency remains integral to the core skills demanded across industries and regions globally. While big data and artificial intelligence (AI) top the list of the fastest-growing skills, there remains a critical need to reinforce foundational technical competencies. One example of a foundational competency is the capability to assemble, maintain, optimize, and troubleshoot personal computers (PCs)—henceforth referred to as PC building. Among other technical know-how, possessing PC building expertise means that individuals can identify suitable components, ensure system compatibility, resolve technical issues, and ensure optimal performance for specific purposes (e.g., gaming, productivity, or general use). The collective ability to perform these tasks reflects underlying competencies such as critical thinking, problem-solving, decision-making, technical knowledge, attention to detail, and adaptability. Mastering these skills prepares individuals for the dynamic and unpredictable nature of technological advancements (Poláková et al., 2023; Tushar & Sooraksa, 2023). With PC building as a gateway to mastering advanced technologies and securing opportunities in a competitive labor market, there is a necessity to develop and implement effective strategies for teaching and assessment in technical education.

Within formal education and workforce development, PC building is best situated within the broader area of computer hardware servicing (Bringula et al., 2022). This field encompasses the installation, configuration, maintenance, and troubleshooting of computer systems and networks, positioning it as a cornerstone of Information and Communications Technology (ICT) programs. As a formally recognized qualification area, this set of competencies is embedded in Technical and Vocational Education and Training (TVET) frameworks that align curricula with occupational standards, certification requirements, and labor market demands. International guidelines such as TVET Strategy 2022–2029 (UNESCO, 2022) and the Global Framework on Core Skills for Life and Work (International Labour Organization, 2021) highlight the need for digital competence, hardware literacy, and problem-solving as essential for workforce readiness in digital economies. Recent TVET literature recommends competency-based whole-institution approaches to digital competence to ensure leaders, educators, and learners all share responsibility for technical skills development (e.g., Zhong & Juwaheer, 2024). In this environment, PC building exemplifies the foundational technical proficiency that supports progression toward advanced ICT qualifications and certification. Framing PC building in this way consequently highlights its dual role as a foundational technical practice and as an integral component of structured pathways toward employability, labor mobility, and lifelong learning in technology-driven economies.

Stealth Assessment and Evidence-Centered Design

Stealth assessment is a covert method of evaluating learner competencies by embedding assessment processes seamlessly into interactive environments. In contrast to traditional assessments, which often disrupt learning and induce test anxiety (Jerrim, 2023), stealth assessment operates unobtrusively by capturing performance data as learners engage with tasks in real time (Shute et al., 2009). This approach allows for continuous monitoring of competency levels and supports adaptive interventions such as adjusting task difficulty or delivering personalized feedback. Importantly, stealth assessment should not be understood as a form of deception but as a means of invisibly collecting gameplay data and applying it formatively to support performance gains, foster self-regulation, and nurture independent learning. Such an approach directly tackles the problem articulated by Shute et al. (2016), particularly on capturing evidence of learning unobtrusively and transforming it into actionable feedback without disrupting engagement. Consequently, several studies have deployed and evaluated the use of stealth assessment in video game contexts (e.g., Fang et al., 2023; Shute & Rahimi, 2021). Despite these efforts, a systematic literature review on the use of video games noted that the development and application of stealth assessment remain in a relatively dormant state (Varghese & Renumol, 2024). This gap suggests the need for further inquiry and broader implementation to unlock the full potential of stealth assessment as a mechanism for evaluating competencies.

A common conceptual framework that provides theoretical foundation for stealth assessment is Evidence-Centered Design (ECD). ECD is a systematic approach to assessment design that focuses on aligning the tasks, observations, and interpretations used in assessments to measure specific competencies or skills (Mislevy et al., 2003). It provides a framework for ensuring that assessments are both valid and reliable by explicitly defining what is being measured, how it is measured, and why the measurements are meaningful (Newton et al., 2021). Although originally developed for educational and psychological testing, ECD has since been applied across diverse domains. At its core, ECD is structured around three interrelated models (see Figure 1). The competency model (or student model) specifies the knowledge, skills, and attributes targeted for assessment. In the context of computer programming education, for instance, this model may include theoretical understanding, algorithmic thinking, logical reasoning, and other proficiencies essential for programming expertise (Garcia, 2024). The evidence model then identifies the behaviors and performances that serve as indicators of those competencies. Such evidence may be drawn from learners' interactions within coding environments, including error frequency, time taken to solve problems, or the creativity of implemented solutions. Finally, the task model prescribes the design and structure of assessment activities that elicit the required evidence. In programming education, this model might include writing a sorting algorithm, debugging existing code, or completing a project with specified functional requirements. Collectively, these models establish a coherent design logic that not only strengthens the interpretive validity of assessment results but also provides a robust theoretical foundation for embedding stealth assessment into complex learning environments.

Video Games, Digital Environments, and Metaverses

According to Rahimi and Shute (2024), stealth assessment is a theoretically-grounded and psychometrically-reliable approach for evaluating, enhancing, and exploring learning processes within technology-enhanced environments. Among interactive platforms, video games (e.g., Fang et al., 2023) and game-based learning environments (e.g., Min et al., 2020) stand out as particularly popular mediums for implementing stealth assessment. A shared characteristic of these mediums is their inherent enjoyment (Caroux & Pujol, 2024) and immersive experiences (Christou, 2014) that foster high levels of engagement and motivation. Such characteristics are critical in learning contexts, as students who are motivated and immersed in the process are more likely to stay focused, persist through challenges, and deeply engage with the content (Bowden et al., 2021; Clark et al., 2016; Li et al., 2024). The engaging nature of these platforms creates a conducive environment where learners are not only entertained but also actively involved in developing and demonstrating their skills. This synergy between immersion and active participation amplifies the effectiveness of stealth assessment, as it seamlessly integrates evaluation with authentic learning experiences. Moreover, by embedding assessment items within gameplay and contextualizing them to mirror real-world challenges, stealth assessment allows students to showcase their skills, behaviors, and competencies in a manner that is both meaningful and reflective of practical applications (Shute et al., 2016).

Building on the affordances of video games and game-based learning platforms, while simultaneously expanding possibilities for interaction, collaboration, and realism, the metaverse represents a promising frontier for implementing stealth assessment. The metaverse can be conceptualized as a networked constellation of immersive and synchronous virtual environments that afford authentic task-based interactions. As a shared digital ecosystem (Garcia et al., 2023; Iqbal & Campbell, 2023), it constitutes an assessment-rich performance space capable of unobtrusively capturing multimodal telemetry of learner actions, decisions, and strategies in real time. These affordances enable stealth assessments to connect observable interaction patterns to latent constructs of competence. Although the metaverse shares with other interactive environments the capacity to deliver immersive and engaging experiences, it distinguishes itself through its persistent and interconnected nature. Rather than being confined to discrete sessions, interactions within the metaverse unfold continuously across sustained digital spaces. Within this ecosystem, games are intrinsic components that draw users to facilitate exploration and interaction (Mohammed et al., 2024). Yet unlike traditional video games, which are typically organized around predefined narratives and objectives (Arif et al., 2024), the metaverse affords open-ended participation. Users can create content, cultivate social relationships, and engage in activities that mirror real-world complexities (Miller et al., 2024; Uddin et al., 2023). In turn, the metaverse supports the embedding of assessment tasks that reflect real-world complexities and provides opportunities for evaluating competencies such as problem-solving, collaboration, and adaptability in authentic contexts (Chen et al., 2023; Sidhu et al., 2024; Singh et al., 2024). Despite its considerable promise, the integration of stealth assessment into metaverse environments remains largely unexplored and represents a gap in current research.

Research Gap and Questions

Despite growing recognition of stealth assessment as a valid approach to evaluating learning, its application remains largely confined to game-based and simulation contexts, with relatively little exploration in metaverse environments. Existing studies have tended to emphasize competencies such as creativity, problem-solving, and collaboration (e.g., Shute & Rahimi, 2021; Shute et al., 2016), but far less attention has been given to technical competencies that underpin employability in technology-driven economies. This imbalance is problematic, given that technical skills are increasingly identified as core requirements for workforce readiness (World Economic Forum, 2025). The absence of empirical research linking stealth assessment, metaverse environments, and technical skills training also leaves a significant gap in both theory and practice. Guided by these gaps, the present study is structured around the following questions:

1. How effective is stealth assessment in measuring technical competencies in a metaverse-based training environment?
2. What is the difference in technical skill development between metaverse-based stealth assessment and traditional assessment methods?
3. To what extent can technical skills acquired in metaverse training be transferred to real-world tasks?

To address the research gaps and answer the research questions, this study is structured as a two-part investigation. Study 1 examines the effectiveness of stealth assessment in measuring technical skills within a metaverse environment (RQ1). This initial phase seeks to establish the validity and reliability of stealth assessment for technical education. Establishing reliability and effectiveness in this context is a critical first step before expanding to broader evaluations. Study 2 builds on these findings by exploring the practical applications of metaverse-based PC-building simulations. Specifically, it investigates their impact on learning outcomes compared to traditional assessment approaches (RQ2) and evaluates the extent to which competencies developed in the metaverse transfer to real-world tasks (RQ3). This two-part investigation is significant because it demonstrates how stealth assessment within metaverse-based technical training can extend theoretical foundations, provide practical guidance for educators, and inform workforce and policy initiatives by linking digital skill development to real-world performance.

Evidence-Centered Design

Competency Model

Adhering to the principles of ECD (Mislevy et al., 2003), an important step in the development process was to construct and validate competency, evidence, and task models. To develop the competency model, three instructors teaching PC building in undergraduate courses were invited to participate in a structured brainstorming session. Consulting domain experts to identify competencies of interest is a common approach in competency model development (Rahimi et al., 2024). During the session, the instructors shared their expertise and insights into the key skills and knowledge required for effective PC building. They discussed common student challenges, compared teaching strategies, and evaluated which skills should be emphasized to reflect authentic practices in the field. The brainstorming process was guided by the metaverse's learning objectives and mechanics, which emphasized building functional PCs in a simulated environment under constraints like budget, specifications, and troubleshooting scenarios. Through this collaborative process, the group identified four key competency domains for the PC Building Competency Model: (1) foundational PC knowledge, (2) strategic decision-making, (3) system assembly proficiency, and (4) diagnostic and troubleshooting. Each competency was carefully defined, justified based on the requirements of effective PC building, and linked to specific game mechanics within the metaverse simulator. These competencies serve as the foundation for evidence, task, and assembly models, ensuring that the simulator's design aligns with both learning objectives and gameplay experiences. Table 1 provides an overview of the competencies, their definitions, and the rationale for their inclusion.

Evidence Model

After coming up with the final competency model, the group collaborated to build the evidence models, which define how observable actions in the metaverse-based training demonstrate the competencies identified. The group identified key observable indicators for each competency, such as correct component selection for foundational PC knowledge, budget adherence for strategic decision-making, and accurate troubleshooting paths for diagnostic expertise. For example, the evidence model for foundational PC knowledge focuses on observable actions that reflect a player's understanding of component roles, compatibility, and specifications. Specifically, evidence is gathered by tracking whether players correctly identify the purpose of components (e.g., selecting a GPU for graphics rendering) and choose compatible parts (e.g., ensuring the CPU matches the motherboard socket type). The evidence model for system assembly proficiency assesses a player's ability to accurately and efficiently assemble a virtual PC by following the proper sequence of steps, ensuring correct placement of components, and making the necessary connections to create a functional system. Observable actions include adherence to the correct order of installation (e.g., installing the CPU before attaching the cooler), accurate placement of components (e.g., ensuring the GPU is securely inserted into the PCIe slot), and proper setup of connections (e.g., connecting the power supply to the motherboard and storage devices). Errors such as skipping critical steps or misplacing components are flagged during the simulation, and scoring is based on the completion rate, order adherence, and the number of corrective attempts required. Figure 4 shows the PC Building Competency Model with examples of observable actions for each competency.

Task Model

With the evidence models defined, the team proceeded to design task models to elicit observable evidence for each competency. These task models were structured to create realistic PC-building scenarios that align with the game's learning objectives and mechanics. The tasks were designed to challenge players in various areas, including component selection, strategic decision-making, system assembly, and troubleshooting. Each task was explicitly tied to the competencies and crafted to ensure a progression of difficulty, allowing players to develop their skills incrementally. Each task incorporated observable actions tied to specific scoring criteria, such as time taken, number of errors, and accuracy of decisions or solutions. These tasks provided players with opportunities to demonstrate their competencies while receiving real-time feedback to support learning and improvement. A comprehensive overview of the tasks, including example scenarios, scoring criteria, and difficulty levels, is provided in Appendix A.

Computational Scoring Pipeline

The stealth assessment system operationalized the evidence models through automated event logging and rule-based scoring algorithms. Each player action (e.g., component selection, assembly order, troubleshooting step) was recorded in log files and matched against evidence rules that specified success, error, or correction. Actions were weighted according to their diagnostic value. For example, correctly matching CPU–motherboard compatibility contributed positively to Foundational Knowledge, while misplacements in System Assembly triggered error codes until corrected. In addition to correctness, the system also considered efficiency and recovery. Time-stamped logs enabled scoring of task completion speed, number of corrective attempts, and sequence adherence. For example, skipping a step in assembly reduced the sequence accuracy score, while correcting the mistake within minimal retries partially recovered points. Similarly, troubleshooting scores reflected both whether the error was resolved and the logical sequence of steps taken (i.e., penalizing redundant or random attempts). Weighted indicators were then aggregated into continuous scores for each competency domain. Each competency score ranged from 0–100, with thresholds aligned to the categorical performance levels defined in Appendix A. This computational pipeline ensured that assessment outcomes were reproducible, transparent, and directly tied to observable in-game behaviors.

Metaverse Application

A key component of this study is the metaverse application in which the stealth assessments are embedded. Developing an entire virtual world is a complex and resource-intensive task. Thus, utilizing an existing platform is a logical approach. Fortunately, the institution hosting this study features an established educational metaverse called the MILES Virtual World (Garcia et al., 2023). This platform provides a versatile foundation for integrating learning tools and assessments into an immersive environment. It has also been previously employed in educational research, underscoring its reliability and pedagogical suitability as a platform for studying learning processes in virtual environments (Garcia, 2025). Initially designed as a digital school environment for social interactions (Garcia et al., 2024), MILES Virtual World has undergone significant updates to support immersive learning experiences. Its latest iteration, the PC Building Edition (Figure 2), marks the first attempt to transform MILES Virtual World into a learning platform. In this version, students can simulate building personal computers by assembling various hardware components. It introduces challenges such as constructing a PC within a specific budget (Figure 3) or designing a PC build tailored to certain specifications (e.g., a high-performance gaming rig). Adding a PC building simulator in the metaverse offers a risk-free environment where students can experiment with configurations without the fear of damaging expensive equipment. The primary objective of the PC Building Edition is to eliminate the reliance on physical hardware while maintaining a hands-on, immersive learning experience.

Study 1: Stealth Assessment in the Metaverse

Methods

Participants and Procedures

The study recruited 40 participants from computing and engineering programs, categorized into novices (limited or no prior experience with PC assembly or troubleshooting) and experienced individuals (at least 6 months of experience in building or maintaining PCs) based on self-reported expertise. Expertise classification was based on self-reported experience duration and prior exposure to PC assembly or troubleshooting tasks. Participation was voluntary, and all individuals provided informed consent. On September 16, 2024, participants visited a controlled laboratory equipped with desktop computers running the metaverse application. After a 15-minute briefing on study objectives, procedures, and the metaverse platform, participants completed a practice session to familiarize themselves with game mechanics, controls, and in-game resources. They then had one hour to complete two random PC-building scenarios: (1) a "Basic Build," which involved assembling a functional PC while focusing on component roles, compatibility, and sequential assembly, and (2) a "Troubleshooting Task," which required diagnosing and resolving three pre-configured issues (Appendix B). The metaverse's stealth assessment system tracked targeted metrics in real-time, while expert evaluators later reviewed participants' virtual builds and troubleshooting solutions using a standardized rubric (Appendix C) aligned with the PC-building competency model. All procedures adhered to ethical guidelines, ensuring participant data was anonymized and used solely for research purposes.

Data Analysis

The data collected were analyzed to evaluate the effectiveness of the stealth assessment system in measuring PC-building competencies. Descriptive statistics were used to summarize participants' performance that are categorized by scoring levels and difficulty levels. Between-group differences (novices vs. experienced participants) for each competency domain were examined using independent-samples t-tests. Effect sizes were computed using Cohen's d to quantify the magnitude of differences, with benchmarks of 0.2, 0.5, and 0.8 representing small, medium, and large effects, respectively. In addition, explained variance (r²) was reported for correlation analyses, and semi-partial r² (sr²) was reported for regression models to indicate unique contributions of each predictor. Adjusted R² was also calculated for overall model fit. Reliability analyses included two approaches. First, the internal consistency of the stealth assessment metrics was evaluated using Cronbach's alpha (α). Second, intraclass correlation coefficients (ICC) were computed in two contexts: (a) a two-way random ICC to assess inter-rater reliability among expert evaluators and (b) a two-way mixed ICC (absolute agreement) to examine the degree of alignment between stealth assessment metrics and expert evaluations. Validity analyses were then conducted. Pearson correlation analysis was conducted to examine the relationships between stealth assessment metrics and aggregated expert evaluation scores. Multiple linear regression was employed to predict expert scores using stealth assessment metrics as independent variables, assessing the predictive validity of the system. Statistical analyses were performed separately for novice and experienced groups to investigate differences in the accuracy and reliability of the stealth assessment system across varying expertise levels.

Results

Reliability of Stealth Assessment and Expert Evaluations

Cronbach's alpha for the stealth assessment metrics was 0.84, indicating strong internal consistency across tasks. Separate analyses by expertise level also showed high reliability, with novices at α = 0.82 and experienced participants at α = 0.87. These results confirm that the system produced stable measurements across groups. The rubric-based expert evaluations likewise demonstrated strong inter-rater reliability, with an overall ICC of 0.86 (95% CI = 0.78–0.91). Subdomain analyses showed consistently high agreement among raters: Foundational PC Knowledge (ICC = 0.83), System Assembly Proficiency (ICC = 0.88), Diagnostic and Troubleshooting (ICC = 0.85), and Strategic Decision-Making (ICC = 0.84). These findings confirm that the expert rubric yielded reliable evaluations across domains.

Participants' performance on the PC-building tasks varied across competencies, difficulty levels, and expertise levels (Table 2). Independent-samples t-tests confirmed that experienced participants consistently outperformed novices across all competencies, with all differences reaching statistical significance and accompanied by large effect sizes. For Foundational PC Knowledge, novices scored 83.5 ± 6.3 compared to 91.2 ± 4.1 for experienced participants (t = –4.58, p < .001, d = 1.45). In System Assembly Proficiency, novices averaged 87.4 ± 7.2 while experienced participants reached 94.6 ± 4.5 (t = –3.79, p < .001, d = 1.20). The largest gap appeared in Diagnostic and Troubleshooting, where novices (70.8 ± 10.4) scored lower than experienced participants (82.3 ± 7.8) on error identification and resolution tasks (t = –3.96, p < .001, d = 1.25). Finally, in Strategic Decision-Making, novices (72.8 ± 11.5) performed significantly lower than experienced peers (81.5 ± 9.0) when making trade-offs under constraints (t = –2.66, p = .011, d = 0.84). The observed significant differences between novices and experienced participants were expected, given the differing levels of prior exposure to PC building. These results provide evidence of discriminant validity, as the stealth assessment system successfully distinguished between groups with theoretically different competency levels.

System–Expert Agreement by Expertise Level

Pearson correlation and ICC analyses were conducted to examine differences in the alignment between stealth assessment metrics and expert evaluations (Table 3). Both groups exhibited strong correlations across competencies, with experienced participants showing slightly higher correlations overall. For novices, the strongest correlation was observed in System Assembly Proficiency (r = 0.74; p = .012), while for experienced participants, Diagnostic and Troubleshooting showed the highest alignment (r = 0.80; p = .010). ICC values were also higher for experienced participants (ICC = 0.84) compared to novices (ICC = 0.77).

Predictive Validity of Stealth Assessment Metrics

The results of regression analyses revealed differences in the predictive power of stealth assessment metrics (Table 4). For novices, the model explained 49% of the variance in expert scores (adjusted R² = .49), with System Assembly Proficiency (β = 0.48; p = .014; sr² = .23) and Foundational PC Knowledge (β = 0.35; p =.002; sr² = .18) emerging as the strongest predictors. For experienced participants, the model accounted for 54% of the variance (adjusted R² = .54), with Diagnostic and Troubleshooting (β = 0.51; p = .012; sr² = .26) and Strategic Decision-Making (β = 0.42; p = 0.003; sr² = .21) contributing significantly to expert scores. Despite these differences by expertise level, the regression models for both groups demonstrated strong predictive validity. Furthermore, it is worth noting that some beta values for novices were higher than those for experienced participants, such as System Assembly Proficiency (β_novices = 0.48, β_Experienced = 0.36). This unexpected pattern may reflect a methodological reason, such as the possibility that novices show more pronounced gains or stronger relationships in foundational competencies as they acquire new skills, whereas experienced participants may display more nuanced performance that is less strongly correlated with specific metrics.

Study 2: Technical Skills and Transferability

Methods

Participants and Procedures

This study recruited two sections of a Computer Hardware Fundamentals class, each consisting of 40 students (n = 80). These sections were randomly assigned to two groups using a computer-generated random number procedure to ensure that each section had an equal probability of being placed in either group. The Stealth Assessment Group (SAG) engaged in a metaverse-based PC-building game with embedded stealth assessment and adaptive feedback, while the Traditional Assessment Group (TAG) learned through conventional methods, such as instructional videos, text-based guides, and quizzes. Efforts were made to balance prior knowledge across groups, with participants self-reporting their PC-building experience. Approximately half of the participants (n = 38) identified as novices (little or no prior experience), while the remainder (n = 42) reported moderate or extensive experience in PC assembly and troubleshooting. To confirm baseline equivalence, independent-samples t-tests were conducted on pre-test scores across the four targeted competencies. No significant differences were observed between SAG and TAG (all p > .05), which indicates that both group of students started with comparable levels of PC-building knowledge and skills prior to the intervention. Both sections were taught by the same instructor, following a standardized syllabus and pacing guide to minimize instructor-related variability. To further control for potential instructor effects, the delivery of lectures, demonstrations, and feedback was scripted and applied consistently across groups. The primary difference between groups lay in the assessment method.

The experiment (Figure 5) commenced during the first week of the trimester (January 2025) and lasted five days, with sessions of two hours per day. From Day 1 to Day 4, students engaged in structured activities and completed PC-building scenarios designed to target specific competencies each day. Each session began with a pre-test to assess baseline knowledge and skills related to the day's competencies and concluded with a post-test to measure learning gains. On Day 5, students from the SAG participated in hands-on testing to evaluate the real-world transferability of skills developed in the virtual environment. They were tasked with assembling a physical PC based on a provided scenario (e.g., gaming or video editing) and troubleshooting a pre-configured issue in a non-functional system. Expert evaluators used standardized rubric aligned with the PC-building competency model to assess component placement, connection setup, and troubleshooting success. Ethical guidelines were strictly followed throughout the study, with informed consent obtained from all participants, and anonymity of data assured.

Data Analysis

The data collected were analyzed to compare the learning outcomes of metaverse-based and traditional methods, as well as the real-world transferability of skills developed in the virtual environment. Descriptive statistics were computed for pre-test and post-test scores for both groups to summarize overall performance across the four days of training. Paired t-tests were performed within each group to compare daily pre-test and post-test scores, with effect sizes (Cohen's d) computed to quantify the magnitude of learning gains. Independent t-tests were conducted to compare post-test scores between the two groups, evaluating the relative effectiveness of the two instructional methods. Metrics from virtual PC-building tasks were also analyzed using independent t-tests to assess the effectiveness of stealth assessment in promoting task-specific competencies. To evaluate the real-world transferability of skills, Pearson correlations were computed between virtual task performance metrics and hands-on PC assembly scores for the SAG, with variance explained (r²) reported to indicate predictive strength. Additionally, where applicable, adjusted R² values from regression models predicting hands-on scores from virtual metrics were examined to assess overall predictive validity.

Results

Learning Outcomes from Pre-Test and Post-Test Scores

Descriptive statistics and paired t-tests were conducted to analyze learning improvements within each group (Table 5). Both groups demonstrated learning gains over the four days of training. However, the magnitude of improvement varied, and differences were not always statistically significant. While both groups showed performance improvements in foundational PC knowledge and system assembly proficiency, no statistically significant differences were observed between their pre-test and post-test scores on Day 1 for the SAG (p = .073) and on Day 2 for the TAG (p = .059). Effect sizes indicated that gains were especially pronounced for the SAG in advanced competencies, with very large effects on Day 3 (d = 1.85) and Day 4 (d = 1.95), compared to more modest effects in the TAG (Day 3, d = 1.20; Day 4, d = 1.40). These results suggest that stealth assessment was particularly effective in supporting the acquisition of higher-order competencies such as troubleshooting and decision-making.

Comparison of Post-Test Scores Between Groups

As shown in Table 6, independent t-tests revealed significant differences between SAG and TAG on Days 3 and 4, where SAG demonstrated superior performance in diagnostic and troubleshooting compared to TAG (p = < .001; d = 1.36) and strategic decision-making (p = < .001; d = 1.39). However, no significant differences were observed on Days 1 and 2, indicating that for foundational skills such as PC knowledge (p = .684; d = 0.09) and system assembly proficiency (p = .761; d = 0.07), both instructional methods were similarly effective. For instance, SAG (19.5 ± 3.4) and TAG (19.8 ± 3.1) achieved comparable post-test scores, suggesting equivalent impact in teaching foundational concepts. These results highlight the stronger impact of stealth assessment on advanced competencies, particularly diagnostic, troubleshooting, and decision-making skills. The observed large effect sizes on Days 3 and 4 reinforce that the metaverse-based training was especially impactful for advanced competencies.

Analysis of Real-World Transferability of Skills

The results indicated strong correlations between virtual and hands-on performance for some metrics, particularly in assembly accuracy (r = 0.82; p = .005) and error diagnosis (r = 0.79; p = .006), suggesting successful transfer of these skills from the virtual to the physical environment (Table 7). Variance explained (r²) confirmed this pattern, with sequential assembly (r² = .67) and error diagnosis (r² = .62) indicating that more than 60% of variance in hands-on performance could be predicted from virtual task performance. For example, students demonstrated high hands-on scores for sequential assembly (92.1 ± 4.8) and component placement (88.4 ± 5.7). However, no significant correlation was observed for budget management (r = 0.20; p = 0.211) and scenario adaptability (r = 0.31; p = 0.097), which may indicate challenges in adapting abstract and context-dependent skills to physical tasks. These results suggest that while procedural and diagnostic skills transferred robustly, strategic decision-making competencies were less easily generalized from virtual to real-world contexts.

Discussion, Implications, and Limitations

Developing hands-on technical skills has traditionally relied on physical practice, but emerging metaverse environments are redefining how such competencies can be learned and assessed. In this study, the metaverse platform was paired with a stealth assessment approach to enable unobtrusive tracking of participant performance during task execution. Participants demonstrated measurable improvements in PC-building competencies, with gains evident across both assembly and troubleshooting tasks, as well as in real-world transfer outcomes following metaverse-based training. These findings suggest that simulation-based environments, when coupled with embedded assessment mechanisms, can facilitate applied learning while enabling its observation in computing education contexts. While both the metaverse environment and the stealth assessment system were integral to the study, they serve distinct roles. The metaverse platform functions as the instructional medium, providing an interactive space in which participants engage in PC-building tasks and develop relevant competencies. In contrast, the stealth assessment system operates as a measurement mechanism, unobtrusively capturing behavioral data such as task sequences, errors, and completion patterns. As such, the observed performance improvements are primarily attributable to the instructional affordances of the metaverse environment, while the contribution of stealth assessment is the enhancement of the granularity and validity of performance measurement.

Stealth Assessment Applications in a Metaverse Environment

To date, the metaverse remains an untapped digital environment for conducting stealth assessments. While prior works have explored the use of the metaverse in education, these studies have primarily relied on self-reported surveys, observational studies, or traditional test-based assessments rather than leveraging in-game performance data for real-time evaluation (Onu et al., 2024; Qian et al., 2023). While valuable, such methods often fail to capture the depth and authenticity of learner interactions within the metaverse. This observation echoes the critique of Shute et al. (2017) that standardized assessments in immersive environments lack the depth needed to evaluate the richness of student learning experiences. Building on this limitation, this study represents an early attempt to apply stealth assessment in a metaverse environment. The findings suggest that stealth assessment is likewise highly suitable for metaverse environments, as evidenced by its ability to measure technical competencies with validity and reliability. This is consistent with earlier studies on stealth assessment in video games (e.g., Abbasi & Kazi, 2020; Fang et al., 2023; Shute & Rahimi, 2021), which emphasizes the capacity of interactive media to unobtrusively evaluate skills and competencies in engaging and authentic contexts. By extending this discreet performance-based approach to the metaverse, the study supports and strengthens the assertion of Rahimi and Shute (2024) that stealth assessment is an effective evaluation tool in technology-rich environments. Overall, the findings illustrate that stealth assessment can be adapted to measure learner behaviors in metaverse environments, which may serve as a useful medium for advancing educational assessment approaches.

Metaverse as a Superior Medium for Technical Skill Development

Consistent with earlier experimental research (e.g., Vate-U-Lan & Cahill, 2024), this study provides evidence that metaverse-based approaches may offer advantages over conventional methods for certain competencies. A recurring theme in the educational metaverse literature is the metaverse's provision of realistic and engaging hands-on experiences as a core benefit over conventional platforms (Onu et al., 2024). Unlike traditional methods, which often rely on static resources like manuals or videos, the metaverse provides an interactive, 3D environment where learners can engage in authentic, task-oriented scenarios (Garcia et al., 2024; Miller et al., 2024; Ng, 2022). For example, MILES Virtual World allows students to virtually assemble a custom PC by selecting and positioning components while receiving real-time feedback on compatibility and performance issues. This hands-on approach mimics real-world assembly processes, allowing learners to practice and refine their skills without the financial or logistical risks associated with physical hardware. Such an approach is particularly advantageous in situations where face-to-face instruction is limited by resource constraints (Rudolphi-Solero et al., 2025; Tong et al., 2024), such as the availability of hardware, safety concerns, or scalability challenges. Unlike metaverse-based environments, in-person training often requires significant financial investment in equipment and infrastructure, can be limited to small groups due to space and resource constraints, and may lack the ability to provide immediate, adaptive feedback tailored to individual learners' needs. These limitations suggest that metaverse environments may provide a practical alternative platform for technical training that would otherwise be difficult or impractical in traditional face-to-face settings (Popov et al., 2024). Similar approaches have been explored in other fields, such as surgical education (Ammendola et al., 2024) and corporate training (Garg et al., 2025), where metaverse-based simulations are being tested for complex procedures.

The scenario-driven environment offered by the metaverse for hands-on training finds its roots in the principles of simulation-based education, which has long been recognized as an effective method for knowledge acquisition and skill development across various disciplines (Vlachopoulos & Makri, 2017). During simulation-based training in a controlled virtual space, students benefit from the safety of an environment where they can experiment, make mistakes, and refine their skills without facing the consequences of real-world errors. In disciplines like medical education, metaverse-based simulation has been shown to accelerate the mastery of technical skills compared to traditional training methods (Popov et al., 2024). One explanation is the capability of the metaverse to seamlessly bridge the gap between theoretical knowledge and practical application. For example, in the MILES Virtual World, students troubleshooting a virtual PC are presented with a scenario where the system fails to boot. They must identify potential issues (e.g., incompatible RAM, missing storage devices, or improperly installed GPUs) by analyzing system error messages, reviewing component specifications, and simulating the replacement or adjustment of parts. This process mirrors real-world troubleshooting tasks while offering the added benefits of experimenting with multiple configurations and resolving complex issues in a risk-free environment that fosters exploration and encourages learning from mistakes. Drawing from research on interactive simulations and immersive learning environments (Hammouda et al., 2025; Miller et al., 2024), manipulating virtual PC components and observing their outcomes in an engaging, lifelike environment fosters deeper understanding and skill retention. Overall, these results demonstrate the metaverse's advantages in replicating real-world contexts while offering flexibility and interactivity to enhance learning experiences.

Boundary Conditions of Metaverse Effectiveness

An interesting and somewhat unexpected result was that traditional instructional methods performed equally well as the metaverse-based approach in developing foundational competencies such as basic PC knowledge and system assembly proficiency. This outcome may be explained by the nature of these competencies, which rely heavily on declarative knowledge (e.g., recognizing component roles and specifications) and procedural repetition (e.g., following a fixed sequence of assembly). Prior research has shown that such lower-order skills can often be acquired effectively through conventional resources such as manuals, videos, and step-by-step guides (Vlachopoulos & Makri, 2017). In contrast, the distinctive affordances of the metaverse (e.g., interactivity, adaptive feedback, and scenario-based challenges) appear to yield greater benefits in domains that demand diagnostic reasoning, decision-making under constraints, and adaptive problem-solving (Hunt et al., 2023; Petersen et al., 2022). This finding contributes to ongoing discussions on blended instructional design, suggesting that while immersive environments may be superior for cultivating higher-order and context-dependent skills, traditional approaches remain pedagogically sufficient and resource-efficient for teaching foundational knowledge (Conrad et al., 2024; Ryan et al., 2022; Santilli et al., 2025; Yang et al., 2024). Future research should examine how hybrid models can strategically align instructional modality with skill complexity.

Real-World Transferability of Skills Developed in the Metaverse

Interestingly, students demonstrated a clear ability to apply skills developed in the metaverse-based PC-building environment to real-world tasks. These findings provide empirical evidence supporting the successful transfer of skills acquired in the metaverse to analogous scenarios in physical contexts. This is a notable strength of the study, as most experimental research on the metaverse tends to focus on comparing methods without evaluating the practical application of the skills acquired (e.g., Vate-U-Lan & Cahill, 2024). Research in similar technology-enhanced learning environments provides insights into this phenomenon. For instance, Dobrowolski et al. (2021) highlighted that the immersive nature of virtual reality (VR) platforms creates an engaging experience, which fosters better skill transferability and practical application. This notion is consistent with the findings of Petersen et al. (2022), who posited that immersive technologies facilitate learning through presence and agency. In augmented reality (AR) environments, Hunt et al. (2023) emphasized that participants' higher engagement levels and the perceived authenticity of the training environment were critical in enhancing their performance. These findings are equally relevant to the metaverse, as illustrated in the MILES Virtual World. For instance, students were challenged to assemble a high-end gaming PC while adhering to a strict budget. The realistic nature of the task enhanced the perceived authenticity of the training by replicating the constraints and decision-making processes learners would face in practical applications. The dynamic and interactive nature of the metaverse further fostered a strong sense of presence and agency, as students could directly manipulate virtual components and immediately observe the consequences of their choices. Building on earlier research, the heightened engagement and realism of the metaverse seem to have facilitated the development of practical skills that were confidently applied in physical PC-building tasks.

Embedded within the training, lifelike challenges, and problem-solving opportunities is the system feedback mechanism of the MILES Virtual World. This mechanism is important because immediate feedback is essential for learning and skill development in the metaverse (Damaševičius & Sidekerskienė, 2024). For example, when students encounter errors in their virtual builds (e.g., attempting to install incompatible RAM or forgetting to connect a power supply), the metaverse provided contextual feedback with detailed explanations to guide their corrections. With this feature, students are empowered to troubleshoot and resolve the issues independently. This aspect connects directly to the tenets of constructivism and self-directed learning (Lee & Ahn, 2025; Li & Yu, 2023). From a constructivist perspective, the metaverse likewise leverages the concept of an "invented reality," where learning is deeply influenced by the learner's ability to perceive and act within the virtual environment (Aiello et al., 2012). Knowledge is not passively absorbed but actively constructed through personal experiences, mental models, and perceptual mechanisms. Whether real or simulated, these processes are intrinsically linked to the learner's interactions with their environment. Moreover, this interplay between knowledge construction and active engagement also reinforced procedural memory (Dubinsky & Hamid, 2024). For example, MILES Virtual World required students to follow precise sequences (e.g., installing standoffs before securing the motherboard) to ensure that procedural steps were practiced and internalized. The combination of immediate feedback, active engagement, and structured practice underscores the metaverse's potential as a promising platform for developing technical skills in an immersive and meaningful way.

Practical, Theoretical, and Policy Implications

From a practical standpoint, this study demonstrates that metaverse-based environments can function as cost-effective, scalable, and pedagogically sound alternatives to traditional hardware-dependent training. In technical domains such as PC building, financial and logistical barriers often limit student access to authentic practice opportunities, as physical hardware is expensive, prone to wear, and requires specialized facilities. By contrast, immersive virtual platforms such as MILES Virtual World enable students to repeatedly practice complex assembly and troubleshooting tasks in a risk-free environment. Errors become low-stakes opportunities for learning rather than costly mistakes, which is a feature particularly valuable for TVET programs operating under resource constraints (Popov et al., 2024). In addition, the embedded feedback mechanisms within the metaverse environment ensure that learning is iterative and adaptive (Baudier et al., 2025), allowing students to refine their skills through immediate corrective guidance. Industry applications are also evident. Technology-driven organizations could integrate such environments into professional training (e.g., Adarkwah & Islam, 2025; Garg et al., 2025; Nofal et al., 2025), enabling employees to rehearse tasks such as hardware maintenance or system troubleshooting in a simulated yet authentic environment. The practical implication is therefore twofold: (1) metaverse-based training may broaden access to skill development by lowering material and financial barriers, and (2) it provides repeatable, scalable opportunities for learners and employees to gain competence through interactive practice.

Theoretically, this study advances the discourse on stealth assessment by extending its applicability to metaverse-based environments. While stealth assessment has been primarily explored in serious games and traditional digital simulations (Abbasi & Kazi, 2020; Shute & Rahimi, 2021), this research demonstrates how it can be embedded into immersive, scenario-driven environments that mirror authentic technical tasks. The application of ECD within the metaverse illustrates the effectiveness of aligning competency models, evidence models, and task models to ensure that assessment is seamlessly integrated into authentic learning activities. This integration supports a shift in assessment theory from decontextualized testing toward process-based evaluation of learner performance (Rahimi & Shute, 2024; Shute et al., 2017). Furthermore, the findings show how metaverse affordances (e.g., presence, agency; Garcia, 2026) expand the range of competencies that can be measured, from procedural accuracy to diagnostic reasoning and strategic decision-making (Chu et al., 2025; Hunt et al., 2023). Operationalizing stealth assessment in this way contributes to theory in two important ways: it illustrates that immersive platforms can capture unobtrusive evidence of learning, and it indicates that metaverse environments may be able to capture not only cognitive skills but also procedural, diagnostic, and adaptive competencies that are less frequently assessed in traditional approaches.

From a policy perspective, the findings point to emerging opportunities for integrating metaverse-based learning and assessment into TVET and higher education systems. Policymakers could consider supporting pilot programs that deploy metaverse-based training modules in resource-limited institutions, particularly where access to physical hardware is constrained. Such initiatives would align with broader policy goals to democratize access to technical education and promote digital equity (UNESCO, 2022). However, the evidence base remains narrow as this study focused on one domain and a limited participant sample. Thus, while promising, policy recommendations must be framed as exploratory rather than prescriptive. Larger-scale, multi-domain studies are needed before widescale adoption. Ethical concerns also warrant policy-level attention. Stealth assessment relies on the continuous collection of learner performance data, raising questions of privacy, data governance, and algorithmic transparency (Damaševičius & Sidekerskienė, 2024). Policy frameworks should consequently establish clear guidelines on how learner data is collected, stored, and used to ensure that educational benefits are balanced against risks to student autonomy and privacy. Overall, these findings suggest that while metaverse-based stealth assessment may offer meaningful opportunities for enhancing technical education, its policy adoption should proceed gradually and supported by ethical safeguards.

Study Limitations and Future Research Needs

While this study provides valuable insights into the use of stealth assessment in a metaverse environment, several limitations should be acknowledged. First, the sample size was relatively small and limited to students from computing and engineering programs. Although novice–expert comparisons provided meaningful differentiation, the modest number of participants constrains the statistical power of some analyses and limits the generalizability of findings to broader learner populations. Future studies should employ larger and more diverse samples, including participants from other TVET domains and higher education contexts. Additionally, real-world transfer was assessed only for the metaverse group. As a result, direct comparisons of transfer effectiveness across instructional conditions were not possible, which limits conclusions regarding the relative impact of the metaverse on real-world performance.

Second, the study focused exclusively on a specific domain that limits the applicability of the findings to other technical skills or fields. Within the broader frameworks of TVET, PC building represents only one of several core competencies. Expanding the scope to include additional modules, diverse disciplines, or multi-skill training scenarios could provide a more comprehensive understanding of the metaverse's capabilities as a learning medium.

Third, while the metaverse environment was designed to closely mimic real-world tasks and physical campuses of the university, it may not fully capture the tactile and sensory experiences of physical PC building, such as handling hardware components or managing spatial constraints. These aspects are integral to authentic skill development in TVET programs (Peter et al., 2025), and their absence may influence the transferability of certain competencies.

Fourth, the metaverse platform used in this study was PC-based. This design decision was intentional to ensure accessibility, scalability, and consistency across participants, since not all institutions or learners have access to VR headsets or haptic devices. While this version required students to navigate the environment using standard computer peripherals, which may not fully replicate the immersive potential of VR, AR, or MR/XR-based metaverse experiences, it provided a practical and cost-effective environment for piloting stealth assessment. Future research could build upon this foundation by integrating advanced hardware such as VR headsets or haptic devices to further enhance realism and interactivity (Liu et al., 2025).

Finally, the short duration of the training and assessment phases may not fully reflect the long-term retention of skills developed in the metaverse. Longitudinal studies situated within TVET frameworks are needed to evaluate the durability of these competencies and their continued application in real-world contexts over time. Addressing these limitations in future research will strengthen the evidence base for integrating metaverse-based learning and stealth assessment into technical education and vocational training pathways.

Conclusion

This two-part investigation provides empirical evidence on the suitability of stealth assessment within a metaverse context and the effectiveness of metaverse-based training in improving learning outcomes and facilitating skill transfer to real-world tasks. Validating the efficacy of stealth assessment in metaverse environments is fundamental to advancing educational technologies that align with modern pedagogical frameworks and industry needs. Beyond this validation, the research offers valuable insights into the broader application of adaptive learning technologies and contextualized assessments. It underscores the importance of aligning ECD principles with immersive environments to create authentic and meaningful learning experiences. Furthermore, the research lays the groundwork for future studies to explore metaverse-based training across other technical and multi-domain competencies, examine the long-term retention and application of acquired skills, and address critical factors such as accessibility, interoperability, inclusivity, and the responsible use of learner data. These directions resonate with ongoing efforts under UNESCO's TVET digital transformation framework and the OECD Learning Compass 2030, both of which advocate for competence-based and technology-mediated education. With significant implications for educators, researchers, and policymakers, this research contributes to the growing evidence base supporting the integration of metaverse environments and stealth assessment in technical education and workforce development.

Appendix A. Sample Tasks, Scenarios, Observables, and Scoring Criteria for PC Building Competencies

The simulator implemented a rule-based event logging system that automatically mapped player actions to competency indicators. Each in-game action (e.g., selecting a CPU–motherboard pair, inserting a GPU into a PCIe slot, replacing incompatible RAM) was logged with timestamps and evaluated against predefined evidence rules. Actions were coded as correct, incorrect, or corrective, and weighted by their diagnostic importance. For example, selecting the correct component role added to Foundational Knowledge, following the correct order of assembly steps contributed to System Assembly Proficiency, and completing logical troubleshooting steps contributed to Diagnostic and Troubleshooting. These weighted values were summed into domain-level competency scores, with performance categories (Excellent, Good, Needs Improvement) reflecting thresholds defined by the scoring criteria below.

Appendix B. Sample PC-Building and Troubleshooting Scenarios

This appendix presents representative examples of the PC-building and troubleshooting tasks implemented within the metaverse environment. These scenarios illustrate the types of activities participants encountered during the study and provide additional detail on task design, objectives, and expected actions. The examples are not exhaustive but are intended to demonstrate how core competencies (e.g., component identification, system assembly, compatibility evaluation, and diagnostic reasoning) were operationalized within the virtual platform. All scenarios were designed to be feasible within a metaverse-based simulation and to generate observable performance data aligned with the study's stealth assessment framework.

Appendix C. Standardized Rubric for PC-Building Competency Evaluation

This appendix presents the standardized rubric used by expert evaluators to assess participants' performance across key PC-building competency domains. The rubric was developed based on the study's competency framework and was designed to ensure consistent, objective, and comparable evaluations of both system assembly and troubleshooting performance. Each competency domain is rated on a four-level scale ranging from Poor (1) to Exemplary (4), with descriptors outlining observable behaviors and performance quality at each level. The rubric was applied to participants' virtual outputs and actions within the metaverse environment, enabling structured evaluation aligned with the study's learning and assessment objectives.