Early discrepancies showed that presenting the whole RoB 2 form encouraged the AI to answer questions that were not active.
This was not a cosmetic formatting issue. In RoB 2, ‘not applicable’ is a routing status. A conditional question becomes active only when a specified earlier response triggers it. An inactive question must be recorded as NA and should contain no evidence extraction or substantive rationale. Answering it can change the response pattern and therefore the domain judgement.This was not a cosmetic formatting issue. In RoB 2, ‘not applicable’ is a routing status. A conditional question becomes active only when a specified earlier response triggers it. An inactive question must be recorded as NA and should contain no evidence extraction or substantive rationale. Answering it can change the response pattern and therefore the domain judgement.
The problem also showed the limit of broad instructions such as ‘follow RoB 2 guidance’. A language model may know the content of a tool and still fail to execute its branching logic consistently. A plausible explanation attached to an illegal pathway remains an invalid assessment. The project response was to stop relying on a flat prompt. The complete tool needed to be parameterised as an explicit sequence: determine applicability, assess evidence only for active questions, apply the official default mapping, record any justified override, and then verify the legality of the route.This became the turning point from prompt engineering to methodological parameterisation.
This was not a cosmetic formatting issue. In RoB 2, ‘not applicable’ is a routing status. A conditional question becomes active only when a specified earlier response triggers it. An inactive question must be recorded as NA and should contain no evidence extraction or substantive rationale. Answering it can change the response pattern and therefore the domain judgement.This was not a cosmetic formatting issue. In RoB 2, ‘not applicable’ is a routing status. A conditional question becomes active only when a specified earlier response triggers it. An inactive question must be recorded as NA and should contain no evidence extraction or substantive rationale. Answering it can change the response pattern and therefore the domain judgement.The problem also showed the limit of broad instructions such as ‘follow RoB 2 guidance’.
A language model may know the content of a tool and still fail to execute its branching logic consistently. A plausible explanation attached to an illegal pathway remains an invalid assessment.The project response was to stop relying on a flat prompt. The complete tool needed to be parameterised as an explicit sequence: determine applicability, assess evidence only for active questions, apply the official default mapping, record any justified override, and then verify the legality of the route.This became the turning point from prompt engineering to methodological parameterisation.
Version 0.2 of the full sequential parameterisation converted the RoB 2 workflow into five separate stages: applicability, evidence, default judgement, final judgement and verification. Each stage was given a specific question and a prohibited shortcut.Version 0.2 of the full sequential parameterisation converted the RoB 2 workflow into five separate stages: applicability, evidence, default judgement, final judgement and verification. Each stage was given a specific question and a prohibited shortcut.The AI could no longer decide that a question applied merely because relevant evidence was present. Applicability had to be determined from the fixed preceding responses. Once active, each signalling question had to be answered independently from its own evidence.
Default domain judgements were then calculated from the official response patterns before any final judgement or override was considered.Failure handling was made explicit. The parameterisation introduced stop or restart rules for template mismatch, illegal routing, unsupported response codes, source contamination, an unidentifiable target result, incorrect judgement mapping and silent overrides. The purpose was not to make the AI sound stricter. It was to make invalid outputs observable and classifiable.Version 0.3 added operational corrections and was checked as an accurate executable transcription of the core RoB 2 routes and default mappings. At that stage, the remaining questions were increasingly about SWAR governance: source locking, model drift, the exact target result, the role of verification and how to preserve the intervention output.Parameterisation did not remove judgement from RoB 2. It separated the parts that can be made deterministic from the parts that must remain evidence-based and explicitly judgemental.