Nadella warns: the AI you pay for is quietly learning your business — and could become your competitor

There’s a quiet tension building beneath the surface of the enterprise AI boom, and Satya Nadella just put his finger on it. In a blog post published Sunday, the Microsoft CEO warned that companies rushing to adopt AI may be paying for their own future competitor — not just in token fees, but in the proprietary knowledge they’re feeding into the models. “You actually pay twice,” Nadella wrote. “Once with money, and once with something far more valuable — the proprietary knowledge you have to disclose to make AI actually work for you.”

The argument cuts to the heart of a growing unease in the industry. Startups and large enterprises feed sensitive data into AI models to get useful results, but in the process, they’re handing over a detailed map of how their business operates. The model provider — whether it’s OpenAI, Google, Anthropic, or any other — gradually learns the contours of the client’s business through every interaction. That accumulated knowledge could one day power a competing product built by the same company providing the AI.

Nadella’s warning isn’t abstract. He spelled out exactly how the leakage happens: models learn from “usage traces.” Every prompt a user types, every tool an agent calls, and — crucially — every correction someone makes when the model gets something wrong. “Each correction eventually becomes experience accumulated inside the enterprise,” he explained. But that experience doesn’t stay inside the enterprise — it stays inside the model.

The asymmetry is what bothers Nadella most. AI companies train on the entire public internet — every blog post, research paper, forum thread, and news article — and call it fair use. But when an enterprise wants to study how a model works in return, the terms of service often shut it down. He called this “ironic” and singled out terms-of-service clauses that “reserve the right to learn from customer usage and interaction data.”

His proposed remedy is model distillation — a technique where you analyze a model’s outputs to understand how it works, then train a smaller, cheaper model based on that knowledge. IT-NEWS explains distillation as a process of feeding a model questions, studying its answers to reverse-engineer its reasoning, and using that blueprint to build a more efficient version. The AI industry has used distillation internally for years, but Nadella argues that the same right should extend to customers who feed their data into these systems.

“You want the model to perform better? You have to give it more knowledge,” Nadella wrote. Competitors can’t buy that hard-won experience at any price — but the enterprise hands it over willingly, one prompt at a time.

The blog post lands at an interesting moment in the AI industry’s evolution. The first wave was about capability — can these models do impressive things? The answer was yes. The second wave, now underway, is about integration — how deeply should these models embed into business processes? The third wave, which Nadella is nudging into view, is about ownership: who actually benefits from the knowledge the model accumulates?

For companies deploying AI today, the question isn’t just whether the model works. It’s whether working with the model is a transaction or a relationship — and whether that relationship has an exit clause.