AI & Enterprise Intelligence
A useful AI pilot is not a production system
A convincing demonstration is a starting point. Production readiness comes from clear acceptance criteria, controlled access and an operating model that can handle failure.
A demonstration answers the question in front of it. A production service has to cope with the next question, the missing document, the changed policy and the person who should never have seen that information. Moving between the two requires a different kind of work.
Consider a hypothetical internal policy assistant. A short trial shows that it can summarise a purchasing policy. That is useful evidence about one task. It says little about conflicting versions, restricted supplier terms or an answer that sounds certain despite missing context. The next milestone should make those boundaries visible.
Define the job precisely enough to judge it
Replace a broad ambition such as improving knowledge access with a concrete responsibility. In this example, the assistant might locate the applicable purchasing rule, explain it in plain language and link to the relevant clause. It should ask for clarification when the business unit is unknown and route exceptions to an authorised person.
Write down what an acceptable answer contains and what the system must decline to do. NIST's Generative AI Profile cautions against extrapolating capability from narrow, anecdotal assessments and recommends verifying sources during testing and monitoring. A successful trial should therefore lead to a sharper testable scope, rather than an assumption of general competence. [1]
Build a test set that can disappoint you
Collect representative questions with the people who understand the work. Include ordinary requests, ambiguous wording, incomplete information and questions the available material cannot answer. Microsoft's evaluation guidance recommends diverse examples that reflect intended use, with measures for individual components and the application as a whole. [2]
Score the qualities separately. A response can be readable while citing the wrong policy, or factually supported while leaving out an important condition. Automated assessment can help compare iterations, but the scoring method needs calibration against human judgment. Google makes that alignment an explicit part of its evaluation guidance. [3]
- Does the answer address the actual task?
- Do its claims agree with the cited material?
- Does it handle missing evidence and restricted information appropriately?
Treat access and data handling as product behaviour
The interface should make the information boundary understandable. Decide who can use the assistant, which sources are eligible and what happens when a person's access changes. Keep those decisions with the application and the underlying information systems; a reassuring sentence in a prompt is not an access-control implementation.
Review the entire data route, including temporary files, external services and diagnostic records. Google Cloud's security guidance distinguishes controls for identity, secrets, sensitive data, networking and audit activity. These are separate responsibilities to design, not qualities automatically supplied by connecting a model. The specific controls should follow the information being handled and the deployment environment. [4]
Operate the chain, not only the model
A slow or incorrect response may originate before generation. Documents can be unavailable, retrieval can select irrelevant passages and an integration can return an incomplete record. Production tracing needs enough detail to distinguish these cases. Microsoft's guidance calls for observing inputs, outputs and intermediate retrieval steps, while Google recommends versioning the components of the application chain. [2][3]
For the policy assistant, that means being able to identify the document version, retrieval configuration and prompt behind a response. Agree on acceptable waiting time and operating cost, and establish who investigates failures. Logging also needs deliberate access and retention rules: debugging should not create an uncontrolled second copy of confidential content.
Make the first release deliberately narrow
Start with a defined audience and a bounded responsibility. An internal assistant that prepares an answer for review is a different release from one authorised to update records or contact a supplier. Expand only when the evidence supports the additional responsibility. This is a release decision, not a feature-count target.
Before rollout, decide how to disable the service, fall back to an existing process and communicate a problem. NIST recommends defined incident responsibilities, deactivation mechanisms and tested fallback arrangements, including manual processing where appropriate. Keep an owner for the service after launch and repeat the relevant evaluation when its sources, configuration or model change. [1]
Sources & further reading
The references behind this perspective. Reviewed . Product documentation can change; check the source before making an implementation decision.
- NIST — Generative Artificial Intelligence Profile, AI 600-1
Primary risk-management guidance. Relevant actions include MS-2.5-001/003 on capability evidence and source verification, GV-1.7-001 on deactivation, and GV-6.2-003/006 on incident ownership and fallback.
- Microsoft — Evaluation and monitoring for RAG applications
Primary documentation for representative evaluation sets, component and whole-system measures, domain-expert feedback and production traces.
- Google Cloud — Deploy and operate generative AI applications
Primary architecture guidance for evaluation aligned with human judgment, versioning application-chain components and end-to-end observability.
- Google Cloud — Generative AI security best practices
Primary documentation separating identity, data protection, secret management, network controls and audit logging within generative AI architectures.
