I’ve come across two interesting reports that highlight how confusing the world of AI is right now — token prices aren’t doing what we thought they would. Adoption also hasn’t yet had the effects we were hoping to see. Perhaps the only good news: An MIT study found that it’s a myth that AI will replace most jobs in the next few years. Read on …

MIT just released a report, here https://nanda.media.mit.edu/ai_report_2025.pdf , that summarizes research into the State of AI in Business in 2025. There is a lot that’s worth looking at in the report —such as how people discover AI technologies (only 20% leverage existing vendor relationships, for example — vendors, take note)— but here are my main takeaways:

• As of the date of publication last month, most AI implementations are replacing business processes that can be easily outsourced and/or moved offshore (called BPO, like payroll or benefits management).
• The study found that of all the verticals the cover (see the note at the end of the post), only technology and media executives anticipate cutting back hiring over the next two years because of AI. It quotes one mid-market manufacturing COO saying that “The hype on LinkedIn says everything has changed, but in our operations, nothing fundamental has shifted. We’re processing some contracts faster, but that’s all that has changed.”
• Not surprisingly, though, big firms lead in pilot and trial implementations — but even they aren’t racing to scale up. They may “allocate more staff to AI-related initiatives … [but] report the lowest rates of pilot-to-scale conversion.”
• The study found that “mid-market companies moved faster and more decisively. Top performers reported average timelines of 90 days from pilot to full implementation. Enterprises, by comparison, took nine months or longer.” 
• While they explore what a custom AI implementation might look like for their enterprise, the study found that over 80% of organizations have explored or piloted ChatGPT and Copilot, “and nearly 40 percent report deployment. But these tools primarily enhance individual productivity, not P&L performance.” 
• “Meanwhile, enterprise-grade systems, custom or vendor-sold, are being quietly rejected. Sixty percent of organizations evaluated such tools, but only 20 percent reached pilot stage and just 5 percent reached production. Most fail due to brittle workflows, lack of contextual learning, and misalignment with day-to-day operations.”
• My takeaway: as with PLM or other initiatives, small, targeted projects that can show a quick ROI are a better bet than bigger, more nebulous ones. Know what problem you’re addressing, how you’ll prove success, and what you’ll do next.

Bottom line for jobs: The study says that 2.7% of “labor value” could be replaced by AI in the near-term, especially in “customer support, software engineering, and administrative functions” because early adopters are seeing “measurable savings from reduced BPO spending and external agency use, particularly in back-office operations … improved customer retention and sales conversion through automated outreach and intelligent follow-up systems.” More sobering is the study’s conclusion that as many as “39 million positions” could be affected by AI, especially as “AI systems develop persistent memory, continuous learning, and autonomous tool integration.

But it’ll be gradual. “Workforce transformation will occur gradually rather than through discrete displacement events … Until AI systems achieve contextual adaptation and autonomous operation, organizational impact will manifest through external cost optimization rather than internal restructuring.” 

All of that is fine but we’re still not seeing any ROI for AI except in limited cases, and most companies aren’t either. The MIT study said that it counts “$30–40 billion in enterprise investment into GenAI,” but that “95% of organizations are getting zero return … Just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact.”

See the study for how they explain the various factors at play. The report is worth your time.

One thing the study didn’t address was the price of all of this. I read a fascinating Substack post last week (https://ethanding.substack.com/p/ai-subscriptions-get-short-squeezed) about the changing cost of using large language models (LLMs). How can you justify buying this technology without looking at its cost?

Like all technologies, AI was initially pitched as expensive for early adopters, and as becoming cheaper as more customers adopt the technology. AI salespeople showed charts of LLM prices dropping 10x with each new version, promising investors fast revenue growth at the beginning and then improving margins as volume picked up. 

The problem: old LLMs did get cheaper, but volume among paying users didn’t rise as expected. As Ethan Ding, the writer of the Substack piece, put it, “gpt-3.5 is 10x cheaper than it was. it’s also as desirable as a flip phone at an iphone launch.” 

The actual cost of using LLMs didn’t follow the expected pattern. Yes, the price of the old LLMs went down, but nobody wanted the old LLM; they wanted the latest, greatest — perhaps rightly so, at the pace these LLMs are improving. But even if the token price for the latest LLM is the same as the original price for the old, consumption went WAY up because the new capabilities needed more processing, meaning each use would be much costlier. Mr. Deng makes up these numbers, but his underlying thesis is not wrong:

“chatgpt used to reply to a one sentence question with a one sentence reply. now deep research will spend 3 minutes planning, and 20 minutes reading, and another 5 minutes re-writing a report for you while o3 will just run for 20-minutes to answer “hello there”. the explosion of rl and test-time compute has resulted in something nobody saw coming: the length of a task that ai can complete has been doubling every six months. what used to return 1,000 tokens is now returning 100,000.”

Let’s put some more or less real numbers against that. I’ve read that some LLMs cost $60-75 per million tokens. At the bottom end of that range, a task that uses 1,000 tokens would cost $0.06. Reasonable. But if that same task plus the added value of the latest LLM now requires 100,000 tokens, we’re at $6. Per task, per user, per day. Is that reasonable? Depends on how many of those tasks and users.

The problem is scaling up use of these LLMs. I’ve read about one user who burned through 10 billion tokens in a month for automated software coding; at this token price, that would cost $600,000. Yes, using the LLM might result in a more efficient solution than a human would create. Yes, it might be error-free (or not — check it!). Yes, the task might be completed faster, since the LLM doesn’t need meal or sleep breaks. But, yikes, $600,000 = sticker shock.

Mr. Deng also writes about the risks to the AI creators of getting this revenue model wrong. Another read worth your time.

TL; DR. There are lots of AI pilots. So far, we’re seeing the most success using AI to support business processes, many of which have historically already been outsourced, so those jobs were already outside the enterprise. Jobs will be lost, just perhaps not as we expect. Only 5% of AI pilots turn into implementations that yield a measurable return. ChatGPT and Copilot are widely adopted, which bodes well for the Copilots we’re seeing in our engineering and design tools — but all of these tools boost individual productivity, which is good but not going to move the needle to $600k/person-or-department/month investment. 

And perhaps most important at the human level: Most AI implementations today aren’t driving headcount reductions. That said, the first impacts are being seen in admin, customer support, where organizations are decreasing their spend on outside agencies. So there might be changes in how those agencies employ people … so, yes, the workforce will be affected.

[Note: The MIT study says that the “Energy and Materials” vertical has “Near-zero adoption; minimal experimentation” and doesn’t call out the big PLMish automotive, aerospace, etc. verticals. I know there is usage/exploration of AI in energy and materials, but put the study’s comment down to a small sample size. As always, read studies like this with a critical mindset — this is directional, not gospel.]