Volume 1 / Research into Practice / Actors · competency assessment
Assessment That Keeps the Instructor in Command: Turning Flight Data into Explainable Learning Evidence
An updated adaptation of competency-based manual-flight research, with a proposed evidence record that connects observed behavior, learner explanation and instructor judgment.
Updated research adaptation · Proposed assessment framework · AI-assisted · Source check: October 30, 2026 · No human peer review recorded
Paper AgentAsk this paper.Explore evidence, limits & sources
Complete journal article
Find passages you can follow back to their source. This reading companion searches this record; it does not invent answers or provide an operational flight briefing.
Source coverage & reuse
Assessment That Keeps the Instructor in Command: Turning Flight Data into Explainable Learning Evidence
Ahmed Allem
This agent indexes the text and editorial record of this journal article.
Open original source ↗ · CC BY 4.0 — new adaptation; source attribution retained
A distinct, updated adaptation
Reframed the licensed assessment study as a new instructional evidence-record proposal; added current FAA ACS context and FITS guidance; developed disagreement-review and contextual-validation considerations. No source model was retrained, and no figures, tables or new experimental results are presented.
Original research: Hong Sun; Xin Zhou; Peiwen Zhang; Xin Liu; Yunsong Lu; Hong Huang; Wuyang Song (2023). Competency-based assessment of pilots’ manual flight performance during instrument flight training. © 2023 The Author(s), CC BY 4.0. Original authorship and DOI are unchanged. Source guide & agent ↗
New adaptation: Ahmed Allem, first author at the commissioning editor’s direction; AI-assisted preparation under Dr. Freedom Cheteni. No new empirical study, source-author endorsement or Ahmed’s personal review is recorded. CC BY 4.0 applies to this adaptation’s new text.
Related product perspective ↗An assessment becomes useful when the learner and instructor can explain what the evidence supports and what it leaves unresolved. This adaptation builds on Sun and colleagues’ scenario-specific approach to assessing manual instrument-flight performance. It proposes a transparent academy evidence record and connects that record to the FAA’s currently listed Airman Certification Standards. The aim is to strengthen learning conversations while keeping an automated indicator within the limits of the evidence used to construct it. [1, 2]
Three ideas to take with you
- Link every indicator to an observable behavior, a task and a stated interpretation.
- Preserve disagreement between a model, an instructor and a learner as evidence to examine.
- Assess the limits of transfer before applying a scenario-specific scoring method elsewhere.
What the sources establish
Sun, Zhou, Zhang, Liu, Lu, Huang and Song developed an assessment method from observable behaviors and flight-parameter indicators, evaluated in a teardrop-pattern procedure. Their model’s assessment correlated with expert evaluation at 0.947 in that study. The participants were male pilot trainees from the same training schools; the sample included 20 trainees selected for excellent evaluations and 30 randomly selected trainees, with three experienced experts. The authors identify unmodeled interdependencies among parameters, the need to reconstruct indices for other scenarios, and the omission of knowledge and attitudes from the skill-focused indicators. The reported correlation is evidence within that evaluated setting, not a general certification standard. [1]
Original analysis & proposed framework
Make the task the unit of interpretation
A flight-data trace becomes meaningful through the situation in which it was produced. The source study constructed indices for a particular procedure; its authors explicitly call for rebuilding the assessment approach for other scenarios. An academy adopting an analogous method should therefore document the task, conditions, equipment configuration and target behavior before computing an indicator. A score detached from these details invites comparisons the research did not validate. The proposed evidence record starts with context, then shows the observation and the interpretation. This structure supports a clearer review of whether the indicator is useful for the task actually being taught. [1]
Use the current standard without inventing a new one
The FAA’s ACS index, checked for this October 2026 update, lists the Private Pilot Airplane ACS-6C and Instrument Airplane ACS-8C editions dated April 2024 and effective May 31, 2024. The index also lists the Flight Instructor Airplane ACS-25. The appropriate task standard gives an academy an external reference for planning instruction and assessment. This adaptation does not create a new FAA threshold or claim that the source model has been accepted for certification. The practical proposal is to identify the applicable task and expected demonstration in the record, then explain how the local observations contribute evidence relevant to it. [2]
Add explanation to the behavioral trace
The original paper’s own limitations point beyond flight parameters: competency also involves knowledge and attitudes. A proposed debrief can therefore pair an observed sequence with the learner’s explanation of what they noticed, expected and intended. The instructor can then compare that explanation with the task record and identify a specific next learning objective. This does not turn a retrospective explanation into proof of every underlying mental process. It adds a distinct source of evidence and makes the difference between observation and interpretation visible. The academy should evaluate whether the combined record improves the quality and consistency of its feedback before using it for more consequential purposes. [1]
Make disagreement reviewable
Suppose a model assigns a favorable score while an instructor identifies a significant contextual concern. The proposed workflow preserves both views and the evidence underlying them. A second reviewer can examine the relevant event, the model inputs, the definition of the indicator and the instructor’s explanation. The resulting record should show whether the disagreement arose from missing context, noisy data, a different interpretation of the task or an unsuitable indicator. This is a hypothetical review process, not a result reported in the source experiment. Its design purpose is to prevent a convenient numerical output from concealing uncertainty that matters for instruction.
Evaluate a model before extending its use
A strong correlation in one study can motivate further evaluation, but the population and procedure still bound the claim. For a new academy, aircraft configuration or scenario, collect evidence that the measurements and interpretation remain appropriate. Examine performance across the learners and conditions actually served, the stability of results when the task changes, and cases where instructors disagree with the output. Avoid presenting a selected study sample as evidence about every pilot population. Any local pilot evaluation should state its sample and missing data, retain human review and distinguish a research indicator from an authorized certification decision. These are proposed safeguards for evaluating use, not findings that the original study established. [1]
Give a learning platform an evidence trail
Aviation Infinity supplies the product-design connection for this adaptation: a learning journey benefits when preparation and feedback refer to the same identifiable objective. The proposed feature is a compact evidence trail linking a concept, a training task, observable behavior, learner reflection, instructor feedback and the next practice step. No claim is made that the platform currently implements the source study’s model or this feature. FAA FITS guidance offers a complementary educational context through scenario-based training, decision making, risk management and single-pilot resource management; FITS describes itself as non-regulatory. A product can support these instructional conversations while preserving the instructor’s responsibility to interpret the evidence. [3, 4]
Build confidence through inspectable decisions
The proposed record ends with a decision and its reason: continue practice, revisit a concept, seek another observation or advance within the academy’s authorized process. Include who reviewed the evidence and what remains uncertain. This gives the learner a concrete route forward and allows the institution to learn from its own assessment decisions. The ambition is an academy that can explain its judgments and improve them over time. Flight data can contribute to that ambition when its role is explicit, its limits are retained and its interpretation remains open to review.
From reading to inquiry
Put the argument to work.
Proposed academy exercise: take one completed, appropriately de-identified training event and create an evidence record with task context, observed behavior, learner explanation, model output if any, instructor interpretation, disagreement review and next practice objective. Evaluate the usefulness of the record before extending it to consequential decisions.
Sources & further reading
- Sun, H., Zhou, X., Zhang, P., Liu, X., Lu, Y., Huang, H., & Song, W. (2023). Competency-based assessment of pilots’ manual flight performance during instrument flight training. Cognition, Technology & Work, 25, 345–356. CC BY 4.0. ↗
- FAA. Airman Certification Standards index. Current index checked October 30, 2026. ↗
- FAA. FAA Industry Training Standards (FITS): non-regulatory scenario-based training guidance. ↗
- Allem, A. Portfolio: Aviation Infinity product and founder account. ↗
Cite this article
Allem, A. (2026, October 30). Assessment That Keeps the Instructor in Command: Turning Flight Data into Explainable Learning Evidence. Aviation Futures: Journal of Flight, Technology & Society, 1. https://journal.research.university/journals/aviation-futures/articles/assessment-that-keeps-the-instructor-in-command-2026
Editorial record
Evidence limits
This adaptation reports no new model fitting, validation or learner outcomes. The original study’s selected sample, scenario dependence and omitted parameter interdependencies remain material. No general validity, automatic certification authority or fairness across untested groups is established. The proposed workflow requires evaluation in its intended context.
Product relationship
Ahmed Allem’s Aviation Infinity relationship is disclosed. The product link is a proposed educational application; no independent product validation, implementation or source-author endorsement is asserted.
Preparation & publication
First-author credit to Ahmed Allem was assigned at the commissioning editor’s direction. Prepared with AI assistance and published by The Research University under the editorial direction of Dr. Freedom Cheteni. Ahmed’s personal review of this AI-assisted text is not recorded. No human peer review is recorded. No DOI or external endorsement is claimed. Source documents retain their original authorship and rights. Proposals and hypothetical examples are identified in the text; this is not an operational flight briefing.