← FAILURE MODES

Wrong-Boundary Automation

Automating commitment where judgment belongs, or inserting humans where determinism suffices

Principle violated

The commitment boundary must sit where task nature, severity, and frequency warrant — not where org chart or vendor defaults place it.

Automation was placed where the framework forbids it — or human gates were placed where they add only latency. The boundary sits at the convenient integration point, the legacy workflow step, or the slot the vendor diagram labelled “approval.” Task nature, severity, and frequency were never applied. The system is automated at the wrong epistemological layer.

In software systems

Two errors mirror each other. Over-automation: an LLM approves commercial credit above materiality thresholds because the API endpoint was wired directly to the commitment event emitter. A rules engine is deterministic and feasible, yet someone replaced it with a fine-tuned model “for flexibility.” Under-automation: tax calculation requires manual spreadsheet review each cycle though the algorithm is closed-form and testable. Fraud scoring for negligible-amount micropayments routes to a human queue because the workflow template defaulted to review-all.

Both misplace the commitment boundary relative to the decision strategy matrix. Over-automation collapses L2 inference into L0 commitment without certification or mandatory human authority at catastrophic severity — often compounding frozen collective subjectivity in the model channel. Under-automation treats deterministic derivation as if it required judgment, burning attention that catastrophic tasks need. Integration convenience — “the model already outputs here” — drives placement more often than epistemic analysis.

In human organisations

Organisations automate politically and staff traditionally. Catastrophic decisions get automated because the vendor promised efficiency and the board wanted AI transformation headlines. Deterministic compliance checks stay manual because a department owns the process and headcount follows workflow steps. LLMs draft and send customer-facing legal responses without counsel at the boundary because marketing needed speed.

The boundary follows budget lines, not task taxonomy. Legal insists on review of everything; engineering auto-deploys what should require certified release authority. Nobody maps tasks to D-F, D-I, or P classification before designing the workflow. The org chart places approval at manager level regardless of severity. Wrong-boundary automation is a staffing diagram mistaken for an epistemological design.

In socio-technical systems

Healthcare again shows the compound case. A diagnostic LLM’s output auto-populates the problem list — commitment without physician boundary — while prior-authorisation for a generic formulary drug routes through a physician portal designed for catastrophic pharmacological decisions. Clinicians drown in low-severity rubber-stamp tasks; high-severity model outputs bypass the attention they require.

Regulatory checklists ask “is there human oversight?” without asking “at the right moment for this task nature?” Vendor RFPs specify “human in the loop” as a boolean. The socio-technical system satisfies audit appearance while inverting the framework’s axiomatic rules: deterministic tasks fed to ML; catastrophic tasks committed by uncertified automation; human attention consumed where machines should decide and absent where judgment is mandatory.

Structural causes

Wrong-boundary automation is primarily a process design failure: the boundary is placed by org chart, vendor template, or integration convenience — not by task nature — severity — frequency. It is not caused by a single anti-pattern slug; it is corrected by applying the decision strategy framework and the commitment boundary at the right layer.

When over-automation also persists inference as fact, epistemic tier collapse and inference-as-fact apply as additional diagnoses.