How to Use AI with Company Documents Without Losing Control of Your Data

Most organizations now have two AI problems at once. Staff are already pasting company documents into chat tools, and leadership wants a sanctioned way to search contracts, policies and reports with an assistant. The answer to both is the same: a retrieval-augmented assistant that runs where you can see it, reads only what the asking person may read, and shows its sources. This guide explains how that works and the decisions that determine whether you keep control of your data.

What a retrieval-augmented assistant is, in plain language

A large language model on its own knows nothing about your business. It was trained on public text and will happily answer a question about your refund policy with a plausible guess. Retrieval-augmented generation, usually shortened to RAG, fixes that by changing the order of operations.

1. The person asks a question.
2. A search layer looks through your documents and pulls out the handful of passages most relevant to that question.
3. Those passages are handed to the model together with the question, with an instruction to answer only from them.
4. The model writes an answer and points back to the passages it used.

The model is never retrained on your documents, and they never become part of it. They sit in a search index you control, and only the passages relevant to each question are sent to the model for that request. This is the pattern behind NeuraCap's Financial Assistant, which answers from filings and shows where each statement came from, and Case Polaris, which retrieves from Canadian legal documents before answering. Every question then becomes a small, inspectable event: what was asked, which passages were retrieved and sent, and what came back.

What leaves your environment, and when

The phrase 'using AI' covers four very different arrangements, distinguished mostly by where your text goes and what the provider may do with it.

Two points matter. First, 'not used for training' is not the same as 'not stored'. Several providers keep inputs and outputs for a limited period, commonly up to 30 days, for abuse monitoring; zero-retention options, where offered, are typically subject to approval. Second, a retrieval-augmented design sends only the passages retrieved for a specific question, never your whole document store. Minimizing what crosses the boundary is a design choice you make, not a feature you buy. Treat the vendor's own data-use documentation as the source of truth and re-read it whenever you change tiers or models.

Canadian data residency: what is actually on offer

Canadian organizations often ask for data to stay in Canada. That is achievable, with some care about what 'in Canada' means.

• The major cloud platforms operate Canadian regions; AWS, for example, lists Canada (Central) and Canada West (Calgary). Your document index, logs and application can run there regardless of which model you use.
• Model availability in Canadian regions varies by platform and model and changes often; some models are served only through routing that may process a request in another region. Check the provider's current regional availability page for the exact model before promising residency.
• Several providers distinguish between where data is stored at rest and where a request is processed. Google's Vertex AI documentation lists a Montréal region but notes that endpoints do not guarantee residency or in-region processing unless chosen accordingly; Microsoft's Azure documentation describes 'Global' deployment types that may process outside the chosen geography; OpenAI's API documentation lists a Canadian residency option with regional storage, subject to approval.
• Self-hosting an open-weight model inside your Canadian cloud account or on your own hardware removes the question entirely, at the cost of running and securing it yourself.

In practice, the achievable target is documents, index, logs and application in a Canadian region, model requests sent under business terms that exclude training, and a written note of where processing happens for your chosen model, re-verified against the provider's documentation at each renewal.

Access control: the assistant must know who is asking

The most common failure in document assistants is not a leak to a vendor. It is an employee receiving a passage from a document they were never allowed to open. Search indexes flatten permissions unless you rebuild them.

• Carry the source system's permissions into the index. Each passage should be tagged with who may read it, and retrieval should filter on the asking person's identity before ranking.
• Authenticate through your existing identity provider (single sign-on) so leavers lose the assistant the moment they lose everything else.
• Keep hard partitions for the most sensitive categories: HR files, board materials, legal holds, client-confidential matters.
• Decide what the assistant may do, not only what it may read; a read-only assistant is far easier to govern than one that can send email or update records.

Audit logs: every question and every source

If someone asks next year what the assistant told a specific person last March, you should be able to answer. Log, with timestamps and user identity:

• The question as asked.
• The passages retrieved, including those filtered out by permissions.
• The exact text sent to the model, and the model and version used.
• The response, its citations, and any feedback or corrections.

Store logs in your own environment with a retention period you set, and restrict who can read them; they contain sensitive content. Logs turn a pilot from a trust exercise into something you can review and defend.

Redaction and minimization: send less

Every passage you do not send cannot leak; minimization is the cheapest privacy control available.

• Index only the document sets the assistant needs; resist pointing it at the whole shared drive on day one.
• Mask personal identifiers (names, health numbers, account numbers, addresses) before passages reach the model where the use case allows; most questions about policies and procedures do not need them.
• Turn off features you do not need, such as cross-session conversation memory or uploads to third-party tools.
• Set retention on your own stores too; documents deleted from the source should leave the index.

Citations, and refusing when the evidence is not there

A document assistant that cannot show its sources is a liability. Two behaviours should be non-negotiable.

First, every factual statement should link back to a specific passage a reader can open. That lets people verify quickly and exposes the cases where the model stitched two unrelated passages together.

Second, the assistant should say plainly when the retrieved documents do not contain an answer. Instruct the model to answer only from supplied passages, set a relevance threshold below which retrieval returns nothing, and test deliberately with questions you know the documents cannot answer. A good assistant says it could not find this in the available documents; a bad one invents a policy.

We hold NeuraCap's Financial Assistant to this standard, and the same rule applies to an assistant over your contracts.

PIPEDA and PHIPA: considerations, not legal advice

We are engineers, not lawyers, and nothing here is legal advice. These are the questions we see privacy counsel ask; good AI design should make them easy to answer.

• PIPEDA applies to private-sector organizations handling personal information in commercial activity and rests on ten fair information principles, including accountability, limiting collection, limiting use and retention, safeguards and openness. A document assistant touches most of them.
• PIPEDA does not prohibit transferring personal information outside Canada for processing. The Office of the Privacy Commissioner's guidance treats a transfer for processing as a 'use', keeps the transferring organization accountable, expects a comparable level of protection (primarily through contract), and expects transparency that information may be processed in another country under that country's laws. A model provider abroad is such a processor.
• Alberta, British Columbia and Quebec have their own private-sector privacy laws; involve counsel early if you operate there.
• PHIPA, Ontario's health privacy law, applies to health information custodians (for example practitioners, hospitals, pharmacies and laboratories) and to their agents and electronic service providers. The Information and Privacy Commissioner of Ontario's guidance notes that PHIPA does not require personal health information to be stored in Ontario or Canada, but the custodian remains accountable and must be satisfied that administrative, physical and technical safeguards are in place, typically through contract. Custodians must take reasonable steps to protect records against theft, loss and unauthorized use or disclosure, and must notify individuals of breaches.
• Whatever regime applies, a privacy impact assessment before the pilot is good practice, and the design choices above are what make it answerable.

Design choices do not guarantee compliance. They are what let a privacy officer look at the system and conclude that it helps meet the organization's obligations.

A sensible pilot

1. Pick one document set with one clear owner and a question the team asks often, for example what your supplier contracts say about termination.
2. Agree the data boundary in writing: region, provider, tier, retention and redaction.
3. Build with permissions, logging and citations from day one; retrofitting them is where pilots stall.
4. Run for four to six weeks with a small group, logging every question whose answer was wrong, unsupported or refused.
5. Review that log with the document owner and privacy lead, then decide to expand, change scope or stop based on what you saw, not on a demo.

Talk to us

If you are weighing an assistant over company documents and want a second opinion on the data boundary, residency or pilot scope, we offer a free 30-minute consultation at /book. We will tell you plainly what is achievable within your constraints.