Back to Blog
Running AI Locally for Confidential Documents: What It Means and How to Choose

Running AI Locally for Confidential Documents: What It Means and How to Choose

•8 min read•meetinginsight.ai
AI & PrivacyBoard Preparation

You want AI's help preparing for a meeting, but the material is confidential — a board pack, a client file, accounts before they're public. Running AI locally means the AI model itself works entirely on your own device, so that material is read, summarised and analysed without ever being sent to an external server. That single design choice is what makes a local AI model — often called a local LLM — usable for confidential documents in a way that AI services such as ChatGPT are not. Choosing one from there is a question of capability and hardware, not confidentiality.

Key takeaways

  • "Local" means the AI model runs on your own device, not on a provider's servers — nothing you feed it is transmitted anywhere, which is what makes it suitable for confidential documents.
  • Concern about AI data privacy is rising fast among business leaders: the share citing it as a top AI challenge rose from 43% to 69% in two quarters, according to KPMG's AI Quarterly Pulse Survey, reported by Cybersecurity Dive, 2025.
  • The best local AI models have nearly closed the capability gap with the largest externally-hosted models — the performance gap narrowed from 8% to 1.7% in a single year, according to Stanford HAI's 2025 AI Index Report.
  • You no longer need specialist hardware to run a capable model: the cost of AI inference fell roughly 280-fold between November 2022 and October 2024, according to the same report, and finished applications now handle the rest.
  • Around half of UK directors cite limited experience with AI or a lack of trust in it as their biggest concern, according to Institute of Directors research reported by Board Agenda, 2025 — the same caution that makes deliberately choosing local processing worthwhile.

What does "running AI locally" actually mean?

A local AI model is one whose files you download once and then run entirely on your own device — a laptop or desktop — with no ongoing connection to the company that built it. The same term is often written as a local LLM, short for large language model: the technology behind ChatGPT and Claude, but running on your machine instead of theirs.

The distinction that matters is not how clever the model is. It is where it runs. A model hosted on external servers requires you to send your documents there to be processed. A local model requires nothing of the kind — the reading, summarising and question-answering all happen on the device in front of you, and nothing travels over the internet to do it.

Why does this matter for confidential documents?

Because what you send to an external server can end up somewhere you did not intend. In early 2023, an engineer at Samsung's semiconductor division pasted proprietary source code into ChatGPT; Samsung banned generative AI tools company-wide soon after, citing the risk that submitted material could be retained on external infrastructure, according to Forbes, 2023.1

Executives increasingly share that worry. The share of business leaders citing AI data privacy as a top challenge climbed from 43% in the fourth quarter of 2024 to 69% by the second quarter of 2025, according to KPMG's AI Quarterly Pulse Survey.2 Among UK directors specifically, around half cite limited experience with AI or a lack of trust in it as their biggest concern, according to Institute of Directors research reported by Board Agenda, 2025.3

The legal exposure is not hypothetical either. A UK Upper Tribunal ruling in 2026 found that putting client documents into an open, public AI tool such as ChatGPT can place that information in the public domain and waive legal privilege — the first English tribunal ruling to address the point directly, according to legal commentary from Norton Rose Fulbright.4 For the full detail of that case, see What Is Local AI? A model that never sends your documents anywhere sidesteps that risk at the root.

Local AI vs external-server AI: what's the actual difference?

FactorExternal-server AI (ChatGPT, Claude, Copilot)Local AI
Where the model runsA provider's external serversYour own device
Where your documents goTransmitted to the provider for processingNowhere — processed on-device
Internet required to useYesNo, once set up
Documents retained by a third partyPossible, depending on the provider's settingsNo
Legal privilege riskPresent, per the 2026 tribunal ruling aboveMaterially lower — nothing leaves the device
Best suited toGeneral, non-confidential tasksBoard papers, client files, and other confidential material

For the full head-to-head, including GDPR and governance considerations, see Local AI vs Cloud AI. And for the specific question a director is most likely to ask, see Is It Safe to Upload Board Papers to ChatGPT?

Is a local AI model as capable as ChatGPT or Claude?

For document work, increasingly yes. The performance gap between the best openly-downloadable models — the kind that can run locally — and the best externally-hosted models narrowed from 8% to just 1.7% in a single year, according to Stanford HAI's 2025 AI Index Report.5 The very largest frontier models still lead on the broadest, most open-ended tasks, but reading, summarising and answering questions about your own documents is squarely within what a local model now does well.

Do you need special hardware to run AI locally?

Less than most people assume, and the trend is moving further in your favour. The cost of running AI at a fixed level of capability fell from around $20 per million tokens in November 2022 to roughly $0.07 by October 2024 — a drop of nearly 280-fold — while hardware costs and energy efficiency have both continued to improve year on year, according to Stanford HAI's 2025 AI Index Report.5 Models have become smaller and more efficient at the same time as ordinary computers have become more capable, which is precisely why local AI has moved from a specialist project to something an ordinary laptop can run.

The remaining question is not the hardware — it's the software wrapped around it. A raw model still needs someone comfortable installing and configuring it. A finished application removes that step entirely: you add your documents and ask questions in plain language, with no model to install and nothing to configure.

How should a non-technical professional choose a local AI tool?

Start from what "local" actually guarantees, and be precise about it. Brian Hengesbaugh, chair of Baker McKenzie's Global Data Privacy and Security Business Unit, cautioned against treating on-device processing as an automatic guarantee of privacy: "I would caution, though, not to let anybody think, 'Oh, great! It's on device. And now we don't have anything to worry about,'" Hengesbaugh told Bloomberg Law, 2024.6 Running locally is the right foundation, but the label alone does not tell you what a given application does with your documents once they're in it.

In practice, that means checking a few things before you choose:

What to checkWhy it matters
Does the model genuinely stay on your device?This is the entire privacy guarantee — confirm it is not a hybrid that sends some material out
Is it a finished application, or a raw model?A finished application needs no technical setup; a raw model requires developer tools such as Ollama or LM Studio
Does it need an internet connection to work, once installed?A tool that must "phone home" to function is not fully local, whatever it calls itself
Does it say plainly what happens to your documents?A trustworthy tool states this directly, not buried in a long terms-of-service document

For a board director or trustee, the practical answer is a purpose-built application rather than a developer tool: something that adds board papers, reads them, and answers questions about them, without asking you to become an engineer to get there. That is the gap meetinginsight.ai is built to close — a local AI model running entirely on your own device, so your board papers are analysed without anything being sent to an external server.

In summary

Running AI locally means the model itself works on your own device, so a confidential document never has to leave it. That single fact is what makes local AI suitable for board papers and client files in a way that AI services relying on external servers are not — and, as the capability gap narrows and the hardware requirements fall, it is no longer a compromise a non-technical professional has to accept to get it.

To see the fuller comparison, read Local AI vs Cloud AI. Or try it directly: meetinginsight.ai runs a local model on your device to analyse board papers, with nothing sent or stored elsewhere. Start a free 30-day trial at meetinginsight.ai/download.

Notes

meetinginsight.ai processes your board papers entirely on your device. Nothing sent. Nothing stored elsewhere. Download a free 30-day trial at meetinginsight.ai/download.

Footnotes

  1. Siladitya Ray, "Samsung Bans ChatGPT And Other Chatbots For Employees After Sensitive Code Leak," Forbes, 2 May 2023. https://www.forbes.com/sites/siladityaray/2023/05/02/samsung-bans-chatgpt-and-other-chatbots-for-employees-after-sensitive-code-leak/ ↩

  2. Eric Geller, "AI security issues dominate corporate worries, spending," Cybersecurity Dive, 26 June 2025, reporting KPMG's AI Quarterly Pulse Survey. https://www.cybersecuritydive.com/news/artificial-intelligence-security-spending-reports/751685/ ↩

  3. Institute of Directors research, as reported by Board Agenda, "IoD: Boards lack AI skills," 29 April 2025. https://boardagenda.com/2025/04/29/iod-boards-lack-ai-skills/ ↩

  4. UK and R (on the application of Munir) v Secretary of State for the Home Department [2026] UKUT 00081 (IAC), as reported by Norton Rose Fulbright, "Court guidance that use of open-source AI waives confidentiality and legal professional privilege," April 2026. https://www.nortonrosefulbright.com/en/inside-disputes/blog/202604-court-guidance-that-use-of-open-source-ai-waives-confidentiality-and-legal-professional ↩

  5. Stanford Institute for Human-Centered AI (HAI), The 2025 AI Index Report, April 2025. https://hai.stanford.edu/ai-index/2025-ai-index-report ↩ ↩2

  6. Brian Hengesbaugh, quoted in "Big Tech Pushing On-Device AI as Privacy, Performance Booster," Bloomberg Law, 20 June 2024. https://news.bloomberglaw.com/privacy-and-data-security/big-tech-pushing-on-device-ai-as-privacy-performance-booster ↩

Frequently Asked Questions

What does it mean to run AI locally?

It means the AI model itself is downloaded once and then runs entirely on your own device, rather than on a provider's servers. Anything you feed it — a document, a question — is processed on your machine and never transmitted anywhere else.

Is local AI safe for confidential board documents?

Yes, when the tool genuinely keeps processing on-device. Because nothing is sent to an external server, there is no query log at a provider, no retained copy on infrastructure you don't control, and nothing that can be exposed if that infrastructure is breached.

What is the best local LLM for someone who isn't technical?

Not a raw model at all — a finished application that packages a local LLM behind an ordinary interface. That gives you the confidentiality of local processing with none of the setup a developer tool like Ollama or LM Studio would ask of you.

Do you need special hardware to run a local AI model?

Less than most people assume. The cost and hardware needed to run a capable model have fallen sharply as models have become more efficient, and a finished consumer application handles the technical side for you — no GPU or configuration required.

Is local AI as capable as ChatGPT or Claude?

For reading, summarising and answering questions about your own documents, yes. The performance gap between the best openly-downloadable models and the best externally-hosted ones has narrowed sharply in the last two years, though the very largest frontier models still lead on the broadest, most open-ended tasks.