An LLM assistant on company data is not “wire ChatGPT to the file server”. Quotes, contracts, bills of materials and internal mail are confidential. Sending them in the clear to a third-party hosted model without a contract, a perimeter and a deletion story is exporting know-how.

The useful question is: where the model runs, which slices of text it sees, and who may ask what.

What it should do (and not)

It helps when people hunt the same fact in ten folders, when onboarding and support repeat the same answers, when sales must find clauses without opening twenty PDFs. It does not help if the process is unwritten: the model does not replace an owner.

It is not an agent that “fixes the ERP”. It is a controlled reader with citations, limits and a log.

Four architecture pieces

  1. Sources. SharePoint, mail, wiki, PDFs, tickets. You need an inventory, not “the whole disk”. Classification and retention before the index.
  2. Index / retrieval. Embeddings and hybrid search (keyword + vectors) on chunks with metadata: customer, date, sensitivity. Without role filters the assistant is a hole.
  3. Model. Local (on-prem GPU or VPC) if data cannot leave. A contracted API only with a DPA, no-training and EU residency, if legal accepts it. A small, fenced model beats an unfenced frontier model.
  4. Application. System prompt, limited tools, citation of the passage, “I don’t know” when retrieval is empty. Audit of queries and answers.

Patterns that work in SMEs

RAG with ACLs. Users only see chunks they could open as files. If they cannot open HR, the index must not return it.

Staging, not production on day one. A closed corpus (manuals and procedures) for weeks. Then you widen. Quality is measured with real questions, not a demo.

No fine-tuning at the start. It is expensive and freezes the wrong habits. Retrieval + instructions + an internal eval set.

Structured output. A summary with sources, not an essay. A link to the document. If there is no source, do not invent.

Expensive mistakes

  • Pasting whole PDFs into context: tokens, noise, leaks.
  • A public site bot on the same internal index.
  • No test environment: a bad query in production is already an incident.
  • “The model is GDPR-compliant” as a magic phrase: the processing is (legal basis, DPIA, processor instructions), not the parameter count.

How to start in four weeks

Week 1: perimeter (which folders, which roles, what is forbidden). Week 2: ingest pipeline and index on a small corpus. Week 3: internal UI, citations, logs. Week 4: eval with ten real questions and a permissions pass.

DPH embeds AI in processes, not brochure chatbots. For a perimeter on your data, book a 20-minute technical session: where documents live, who uses them, whether the model may leave the network.