Human Oversight That Earns Its Place: Testing Review Quality in a Family Office

A signature is not an outcome measure

A family office may place a human approval step after an automated recommendation and describe the workflow as controlled. That step establishes who is authorized to act; it does not establish that the reviewer recognizes errors or improves the recommendation. The important operational distinction is between having authority to intervene and demonstrating an ability to intervene usefully. Both can matter, but they require different evidence. A signed approval record is evidence of authorization, while an independently scored comparison of decisions is evidence about quality.

Sele and Chugunova provide a reason to test that distinction. Their experiment varied whether participants fully delegated a prediction or could adjust a recommendation. The results do not imply that a family office should remove people from investment, payment or fiduciary decisions. Human judgment can serve accountability, context, values and escalation purposes that a single prediction-error measure does not capture. The narrower lesson is that the label human oversight should not substitute for evaluating how the actual review process performs.

What the experiment establishes, and what it does not

The online experiment involved 292 participants drawn from a university-associated pool in Switzerland, with an average age of 24. Participants predicted a student’s percentile score on a mathematics test from a short profile. They could select a recommendation attributed to another person or to a statistical model. Recommendations were curated to have similar accuracy; participants were told that average accuracy was equal. This is a deliberately controlled prediction environment, not an examination of professional investment committees or modern language-model systems.

In the delegation condition, algorithmic recommendations were chosen in 66 percent of source choices. Allowing adjustment increased that preference by seven percentage points. Yet average absolute prediction error was larger when adjustments were permitted: 18.0 rather than 17.4 percentile points. Confidence also increased. These findings distinguish an increase in acceptance from an improvement in accuracy. The observed numerical difference belongs to this task and sample; it cannot be translated into a portfolio-return forecast or a universal error rate for AI.

Participants were less likely to adjust recommendations containing larger errors, and adjusted those errors by less when they intervened. This challenges the assumption that a human reviewer will reliably act as an emergency brake. It does not prove that trained professionals, additional information, feedback or a different interface cannot improve review. The experiment supplied no ongoing accuracy feedback, and it used a narrow prediction task. Its discussion of legislation reflects the publication period, not current legal advice. The protocol below is an original editorial proposal that the researchers did not test.

Define one workflow and its decision rights

The chief operating officer should choose a bounded workflow before evaluating oversight. An example is extracting debt maturities from loan documents for a liquidity dashboard. Specify exactly what the system produces, which documents are authoritative, what the reviewer can change, and which decisions remain outside the tool. Do not begin with an unrestricted assistant that can approve payments or place trades. A test of document extraction cannot justify authority over investment execution.

The workflow owner should document why human involvement is required. Accuracy checking, unusual-case escalation and authorization are separate objectives. Assign a named responsible person to each objective and define what evidence would show that it is being met. If a reviewer is expected to catch a missing covenant, provide access to the governing document and sufficient time to examine it. A reviewer who can see only the model’s summary cannot verify information that the summary omitted.

Compare the system and the reviewed output on the same cases

The risk lead should assemble a test set with independently verified answers, using authorized historical or synthetic documents. Include routine cases and difficult cases, such as amendments that override an earlier maturity date. Keep the reference answers hidden from reviewers during the exercise. Record the original automated output before any adjustment, then record the reviewed output separately. Preserving both makes it possible to measure whether intervention corrected an error, introduced one, or left it unchanged.

Test cases should represent the intended workload and include deliberately selected serious-error examples. Report the representative sample separately from the challenge set so that an enriched collection of difficult cases is not presented as the everyday error rate. Define materiality in operational terms before examining results: a wrong borrower name and an incorrect payment deadline may have different consequences. A specialist independent of the workflow should resolve disputed reference answers. Where the underlying document is ambiguous, mark the case unresolved instead of manufacturing a clean benchmark.

Evaluate correction quality, not correction volume

The evaluation lead should report the system-only error rate, final reviewed error rate, serious errors detected, serious errors missed and errors introduced by review. Review time and reviewer confidence are useful secondary measures. Neither a high override rate nor a low override rate is inherently desirable. Frequent intervention can reflect useful scrutiny or unnecessary changes; infrequent intervention can reflect an accurate system or weak monitoring. The outcome comparison is needed to distinguish those possibilities.

Examine results by document type and by reviewer experience where the sample supports it. Avoid claiming that subgroup differences are stable when there are only a few cases. Have reviewers record a short reason for each substantive change, tied to a passage in the authoritative document. That creates an auditable explanation and helps identify whether an apparent improvement arose from new information, correction of a parsing error, or an unsupported intuition. Do not reward reviewers merely for agreeing with the system or for producing more interventions.

Use a limited release with explicit stopping conditions

The authorized committee should approve a limited deployment only after the workflow meets predefined acceptance criteria. Those criteria are local governance choices, not thresholds supplied by this research. They should reflect the cost of missed errors, the reversibility of decisions and the availability of independent checks. For a sensitive workflow, keep the system in a non-executing advisory role and require a separate authorized decision. Preserve a manual fallback that staff can actually use.

Stop or restrict the workflow if serious errors pass review, if reviewers cannot access source documents, if an input format changes without validation, or if the model version changes materially. Investigate whether the failure arose in extraction, presentation, reviewer knowledge, workload or authority design. Retraining the person alone may not address an interface that hides crucial evidence. Conversely, replacing the model alone may not correct unclear responsibility. Repeat the comparison after material changes before restoring broader use.

Retain a learning record without claiming proven returns

The operations secretary should retain the workflow specification, test-set provenance, reference-answer process, model version, original outputs, reviewed outputs, correction reasons, acceptance decision and subsequent incidents. Periodic review should sample both approved and escalated cases. Reviewing only the cases that people noticed would miss failures of detection, the concern highlighted by the experiment. The board should see whether human review is adding measurable quality and where accountability requires a different arrangement.

This protocol is an AI-assisted editorial analysis for family-office governance, not a tested intervention or individualized investment or legal advice. The original study’s authors are not endorsers of this proposal. No new participant data, independent human acceptance or improvement in investment returns is asserted. The practical objective is to make oversight claims testable: preserve human responsibility while measuring whether the specific review process actually helps.

Source, attribution and limits

[1] Daniela Sele; Marina Chugunova. Putting a human in the loop: Increasing uptake, but decreasing accuracy of automated decision-making. PLOS ONE (2024-02-09). https://doi.org/10.1371/journal.pone.0298037. CC BY 4.0.

N=292, mostly young university-associated participants; narrow prediction task, curated similar-quality human/model recommendations and no ongoing feedback. No direct family-office, investment-return or LLM evaluation; oversight effects are context-specific. Publication-era legal discussion is not current legal guidance.

Prepared by the Family Office Private Wealth Journal editorial research desk with AI assistance at the direction of Dr. Freedom Cheteni. Original research authors retain full credit and are not authors or endorsers of this companion. This proposed process has not been evaluated in a new empirical study; no independent peer-review acceptance is claimed. This is educational analysis, not individualized investment, legal or tax advice.

Implementation record

Open the cited sources

1. [1] Daniela Sele; Marina Chugunova. Putting a human in the loop: Increasing uptake, but decreasing accuracy of automated decision-making. PLOS ONE (2024-02-09). ↗