AI knowledge management usually rests on one belief: if the AI reads the right documents, it will give the right answers. A 2025 Stanford evaluation tested that belief on legal research AI tools marketed as "hallucination-free", which answer only from a store of documents. They still produced false information 17% to 33% of the time.
A company's own documents are rarely kept as carefully as a legal database. Procedures describe the normal case, many versions are out of date, and few have a named owner. Everyday questions get fast, fluent answers and new staff ask less. But the cases where a company loses money are the ones that do not fit the procedure, and there the AI gives advice that is right on paper and wrong on the shop floor. The person who gets called is still the experienced employee.
An AI that has learned every procedure does not make strong people redundant. It changes their job. This article explains why having documents is not the same as being able to do the work, the risk of an AI that can explain but cannot do, and four management decisions that keep what AI looks up correct.
Having documents is not the same as being able to do the work
The first article in this series looked at why good people hesitate to share what they know. This one starts from the next step: suppose they have shared, and the documents are all loaded. What is still missing? Five words that are often used as if they meant the same thing help answer that.
| What it is | Plain meaning | Example from one order |
|---|---|---|
| Data | Recorded numbers and facts, not yet read for meaning | Order for 5,000 units, delivery on the 20th |
| Information | Data placed in context so it says something | The line can make only 4,000 by the 20th |
| Knowledge | Tested understanding of why, and of what to do | When capacity is short, split the delivery and agree it with the customer first |
| Capability | Being able to do it under real conditions: time, authority, contacts | Knowing which customer accepts a split and who to call there |
| Judgement | Choosing well when the rules are not enough or conflict | Deciding that this customer matters more than this month's cost target |
Documents are the base. Each level above needs something documents do not hold.
An AI loaded with documents works well on the first three rows. It finds data, puts it in context and repeats knowledge someone wrote down. Capability and judgement depend on the situation at hand, the authority to act and accepting the consequences. That is where AI knowledge management meets its limit, and where experienced people still carry the work.
The limit is not the same everywhere. In highly repetitive work, such as answering questions on a returns policy, documents cover almost every case and AI can carry most of the load. The gap matters where exceptions are frequent and expensive: production, purchasing, after-sales, key accounts.
Procedures cover the normal case; exceptions are where money is lost
A procedure is written for the case that happens most often. Exceptions are the cases where its assumptions no longer hold: a different material, a tighter tolerance, a machine running below its usual speed. Field studies of work have long found the same gap. The way people actually work usually differs from the way manuals, training courses and job descriptions describe it, and the difference is often where the useful learning happens.
Common signs: manufacturers load hundreds of procedures into an AI assistant. When a special order comes in, with a substitute material, a tighter finish and a short lead time, the AI returns the standard routing and standard times. It cannot know that one machine has been unstable with that material, or that this customer accepts split deliveries. The production director hopes to depend less on the senior team leader; the new planner trusts the AI because its answer is complete and well written; the team leader knows where the procedure is wrong, but nobody asks, and writing it down is not part of the job. Each role acts sensibly. The risk sits in the gap between them.
What makes an exception manageable is often known on the shop floor, not in the procedure file.
The obvious fix is to write every exception into the procedures. That has a cost: procedures grow long, contradict each other, and get harder for people and AI to use. A more practical split is to write down the exceptions that recur, and for rare ones, teach the system to recognise "this is not a normal case" and route it to a person.
Old, conflicting and ownerless documents: the weak point of AI knowledge management
To see why old documents matter so much, it helps to be precise about how AI "learns" company procedures. In most company deployments it does not learn them in the sense of being trained. When someone asks a question, the system searches the document store, picks the passages that look relevant and writes an answer from them, much as a new employee would flip through the handbook. This is called retrieval-augmented generation (RAG). Training is a different thing: it changes the model itself.
| Looking up documents (RAG) | Training the model | |
|---|---|---|
| Where the AI's knowledge comes from | The company's documents, read at the moment of the question | Patterns built into the model during training |
| When a procedure changes | Fix the document; answers change immediately | Retrain; slow, costly, hard to fix one detail |
| Can it show where an answer came from? | Yes, if documents carry a date and an owner | Generally not |
| Main risk | An old or wrong document gives an old or wrong answer | Out-of-date knowledge that cannot easily be traced |
Most company AI reads documents when asked. The quality of the answer follows the quality of the document.
The research behind this approach states the trade-off plainly: knowledge stored inside a model is hard to update and hard to trace to a source, so the method keeps knowledge in an external document store. For a CEO, this means the AI's answers are only as good as the documents. Keeping them correct is no longer an archive task; it is part of running the business.
Three problems then become visible. Out of date: procedures are written once, but work keeps changing, and nobody's job is to retire the old version. Conflicting: two departments write two versions of the same step, for example export packing before and after a customer changed its labelling rules. A long-serving employee knows which one is dead. The AI does not; it may find both and blend them. Ownerless: when a document is wrong, nobody is responsible for fixing it, so it stays wrong.
The trade-off is time. The people best placed to say which document is still right are the same experienced people who are already busy. A small company with a few dozen stable procedures does not need a heavy AI knowledge management system. A company whose products, customers and rules change every quarter does.
In AI knowledge management, experts move from doing the work to setting the rules and checking
The best evidence on where AI helps and where it hurts comes from a field experiment with 758 consultants at Boston Consulting Group, run by researchers from Harvard Business School and other universities in 2023. On 18 realistic tasks that were within what the AI could do, consultants using it completed 12.2% more tasks, 25.1% faster, with clearly better quality. On one complex task chosen to be just outside what the AI could do, those using AI were 19% less likely to reach the correct answer than those working without it.
The same tool helps on some tasks and hurts on others. Source: Dell'Acqua et al., Harvard Business School working paper, 2023.
The researchers call this a jagged frontier: AI is strong on some tasks and weak on others that look just as hard, and the border is not visible from outside. Someone has to know which side of the border a task sits on. That person is the expert. In practice the role has four parts:
- Decide what goes to AI: which questions AI can answer alone, and which must go to a person.
- Write the rules and the exceptions: not the whole procedure again, but the conditions under which it stops being true.
- Check samples of AI answers against how the work was really done.
- Handle new cases and turn the ones that recur into rules.
This is the answer to the question in the title. AI knowledge management does not remove the need for good people; it changes what they spend time on. The trade-off is real: hours spent checking AI are hours not spent on orders, so the role works only when it counts in workload and evaluation. And if one expert is the only checker, the dependency on one person has simply moved.
The risk of an AI that can explain but cannot do
An AI answer sounds equally confident whether it is right or wrong. Managers and new staff tend to judge it by how complete and fluent it is, because that is what they can see. This is the core of the "can talk but cannot do" risk: a clear explanation is not the same as the ability to carry out the work, and the AI does not see the machine, the customer's mood or the cost of being wrong.
Looking up documents reduces this risk but does not remove it. A peer-reviewed study published in 2025 tested leading commercial AI legal research tools whose providers claimed that retrieving sources made them free of invented answers. The researchers found fewer errors than a general chatbot, but the tools still gave false information between 17% and 33% of the time. Legal research is not production planning, but the lesson carries over: having the right documents does not guarantee the right reading of them.
For example, asked how to handle a quality complaint, the AI drafts every step but proposes replacing the whole batch, while this customer's contract covers only the faulty units. The simplest test is to replay past cases the company handled well and compare the AI's answer with what the expert did. In low-stakes look-ups the risk is small; where money, customers or safety are involved, it is the main risk.
A fluent answer is not a checked answer. Checking still needs someone who knows the work.
A feedback loop from real work keeps AI knowledge management alive
A document store left alone gets worse every month, because the work changes and the documents do not. The only reliable source of updates is the work itself. Each time someone departs from the AI's guidance, or handles a case the documents missed, the company learns something, and usually loses it because nobody writes it down.
AI answers from documents
Staff handle the real case
Note the difference in three lines
Owner reviews and decides
Document updated or exception added
Each case the documents missed becomes an update, instead of a lesson only one person remembers.
The note should be short: what the case was, what the AI said, what was actually done and why. The owner of that area reviews the notes on a fixed rhythm and decides whether to change the document, add an exception or leave it as a one-off. The cost is extra writing, the "more work" fear raised in the first article, so a note must take minutes. The loop also fails quietly if nobody reads the notes: people see nothing change and stop writing.
Four management decisions for AI knowledge management
A list of knowledge sources, each with an owner
List what the AI may look up: procedures, price rules, product specifications, contract terms. Each source has one named owner who answers for it being correct. A document with no owner should not be used by AI.
Versions, review dates and expiry
Every document carries a version, a date and a review date. When a new version is approved, the old one is withdrawn, not left beside it. A document past its review date is flagged until its owner confirms it.
Test on real cases and exceptions
Before relying on AI in an area, collect a set of past cases, including the awkward ones, and compare the AI's answers with what experts did. Repeat the test whenever the documents or the AI change.
Connect AI to live business data, within authority
A procedure says what to do in general. The order, the stock level and the customer's history say what is true today. AI that reads both gives more practical answers. Who may see what is the subject of a later article.
AI knowledge management is not a project to load documents once. It is the ongoing work of keeping what the AI reads correct, and that work belongs to the people who know the business best.
Review dates and named owners turn document upkeep into a regular management task.
Points to keep in mind
AI will not absorb all tacit knowledge. Better documents and a feedback loop narrow the gap but do not close it. Judgement in new situations and personal relationships stay with people.
Do not confuse looking up with learning. When someone says the AI "has learned" your procedures, ask whether they mean look-up or training. Usually it is look-up, which is good news: fixing documents fixes answers.
The evidence has limits. The consultant experiment used one AI model in 2023 and tasks designed for the study; newer models may push the frontier further. The legal study covers one field. Neither was done in Vietnam. They show a pattern worth managing, not a forecast for your company.
Do not build more than you need. A small company does not need a knowledge department. Start with the twenty questions staff ask most and the areas where exceptions cost the most.
The direction Viindoo has chosen
An AI that has read hundreds of procedures can still plan a special order wrongly, because it does not know that a machine is unstable with a substitute material, or that a customer accepts split deliveries. That knowledge is not missing from the documents by accident: it lives in daily work, and nobody has made recording it part of the team leader's job.
Viindoo therefore treats AI knowledge management as part of running the business, not as a separate library. Its direction is an Agentic AI business management platform where procedures, experience, operational data and AI sit in one system, so knowledge stays next to the orders, stock and production it describes, and the company decides who keeps each piece of it.
That matches the four decisions in this article. Each knowledge source has an owner, a version and a review date, like any other process. AI answers can be checked against live data, such as which machine is running and what this customer has accepted before, within each person's authority. And the team leader's role shifts from carrying everything in his head to deciding exceptions and updating the rules, so AI knowledge management has someone accountable for it.
For the new planner, asking AI should take no more effort than asking a colleague: no need to know where a document is kept or which version is the latest. Viindoo is the backbone underneath, keeping knowledge, data and access together, so answers are easy to get and also trustworthy.
No platform can capture judgement that has never been written down. AI knowledge management still needs the feedback loop from real work and the people described above; a platform only keeps that loop from breaking.
Four questions to check where you stand
- Who confirms that a document the AI reads is still correct, and when did they last do it?
- Can the AI show which document and which version an answer came from?
- When someone handles a case differently from the AI's guidance, where does that lesson go?
- Which tasks sit outside what the AI does well, and who in the company knows where that line is?
Frequently asked questions
Usually not. Most company AI looks up documents when asked and answers from them (RAG). Training changes the model itself. With look-up, fixing a document fixes the answer, which makes document upkeep the main lever.
Yes, in a different role. Procedures cover the normal case. Experts decide what AI can handle, write the exceptions, check AI answers and deal with new cases. A large experiment with 758 consultants found AI users did worse on a task just outside the AI's ability.
It reduces errors but does not remove them. A 2025 peer-reviewed study of commercial AI legal research tools that retrieve sources still found false information in 17% to 33% of answers.
It depends on how fast the area changes. Stable policies can be reviewed yearly; product, price and customer rules may need a quarterly review, plus a review whenever feedback notes show the same gap more than once.
Next in this series: once AI is connected to the company's documents and data, it starts to draft and suggest real work. When one of those suggestions is wrong and someone acts on it, who answers for the result? When AI gets it wrong, who is accountable?
Sources
- Dell'Acqua, F., McFowland III, E., Mollick, E., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School Working Paper 24-013. 758 BCG consultants.
- Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies, 22, 216-242.
- Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
- Brown, J. S., & Duguid, P. (1991). Organizational Learning and Communities-of-Practice: Toward a Unified View of Working, Learning, and Innovation. Organization Science, 2(1), 40-57.
