Who can let the AI assistant reply to a trading enquiry

A wrong description is inconvenient. A wrong price, quantity, delivery date or payment term can cost money and damage a relationship. The assistant may find an old quotation, treat an indicative supplier date as confirmed, or miss that the customer's requested quantity exceeds available stock. Its prose can still sound entirely certain.

The practical boundary is between preparing information and making a commitment. An assistant might acknowledge receipt, identify missing details, retrieve an approved product description and prepare a quotation draft. Sending a binding price, confirming stock, changing a customer record or promising a delivery date should require an authorised employee unless the business has deliberately defined a narrower, reliable rule for that action.

That distinction makes governance concrete. Instead of asking whether AI is allowed, the business asks which actions it may take, using which records, under whose authority, and with what evidence.

Write down the ordinary path. An enquiry arrives by email or a form. Someone identifies the customer and product, checks quantity and location, looks up current stock or asks a supplier, calculates the commercial terms, drafts a response, gets approval when needed and records the outcome. Exceptions go to a person who can resolve them.

Now mark the information each step depends on and where it lives. Product details may come from a maintained catalogue. Stock may be a live system, a spreadsheet updated twice a day or a supplier confirmation. Those sources are not interchangeable. A model should not present a stale spreadsheet as live inventory simply because it found a number.

The map also reveals where automation is premature. If staff do not agree how to handle partial availability or substitute products, a model cannot infer a stable policy from scattered examples. Resolve the operating rule first, then let software carry it out.

A first version might read approved product and customer information, draft a response and create a review task. It need not send mail, edit negotiated terms or access every shared folder. If the task works with read access and a draft, begin there. Expand authority only when the need and safeguards are understood.

Approval should attach to meaningful changes, not become a ritual button. A routine acknowledgement may be sent automatically from an approved template. A quotation above a chosen threshold, an unusual discount, a delivery promise without supplier confirmation or a change to payment terms should reach the responsible employee. The business sets the thresholds; the model does not invent them.

The reviewer needs enough context to make a decision quickly: the original request, the source of the stock and price information, any uncertainty, the proposed reply and the exact action that will follow approval. If approval merely says 'AI wants to send an email', staff will either approve blindly or abandon the process.

A customer can ask for a price and also include text that tries to redirect the assistant: ignore previous rules, reveal another customer's terms, or send the entire account file. That text is part of the enquiry. It must not become a new instruction with authority over the application.

OWASP's LLM application guidance identifies prompt injection and sensitive information disclosure among the risks developers should address. For this workflow, useful controls include separating trusted system rules from customer content, limiting what retrieved documents contain, restricting tools to the task, checking proposed actions in application code and requiring human confirmation for consequential sends or updates. A clever prompt is not a permission boundary.

Data access also deserves a practical review. Decide which customer fields the assistant actually needs, where prompts and outputs are processed, what the service retains, who can inspect logs and how long records are kept. Those choices depend on the selected tools and the business's obligations, so they should be checked against current provider terms and applicable requirements before rollout.

For each assisted enquiry, retain the request, relevant source references, draft, review decision, final message and any system action. Record the human approver and timestamp where appropriate. Avoid logging unnecessary personal data just because storage is cheap. A useful log makes a dispute or correction understandable without turning into a second uncontrolled customer database.

NIST's AI Risk Management Framework organizes work around governing, mapping, measuring and managing risk. Applied here, that means assigning ownership, understanding the workflow and its context, checking performance against defined criteria, and responding when the system falls outside them. The framework provides a way to structure the work; it does not certify a particular assistant as safe.

Plan for ordinary failure. The catalogue may be unavailable, a supplier may not answer, the model may produce an unsupported field, or an employee may spot an incorrect recommendation. The system should stop or route the case to a person, preserve the original enquiry and make correction straightforward. A quiet fallback to an invented answer is not graceful degradation.

Choose a narrow enquiry type and a small group of users. Before adding the assistant, record how long the current path takes, how often replies need correction, how many enquiries wait for missing information and how frequently prices or dates are revised. These are baseline measures, not promised gains.

During a pilot, compare the same measures alongside approval time, unsupported statements, escalations and employee effort. Review a sample of outputs, including rejected drafts and cases that fell back to manual handling. A faster draft is useful only if the final response is accurate and the full task takes less effort without pushing risk onto the customer or reviewer.

Set a pause condition before launch. For example, stop automatic sending if a source becomes stale, a threshold is crossed or an unsupported commitment appears. The team should know who can disable the feature and how enquiries continue while it is paused.