Estimated reading time: 10 minutes

What’s the payback time for your fancy GenAI project? Will the rate of return outdo the stock market? As GenAI becomes less magical fairydust, and more hands-on work, it’s time you looked at the ROI on your GenAI initiatives.

A friend in Seattle started working on a b2b application two years ago. The idea was decent, and he had done fair bit of market research. They released the beta version last year. Sales was slow, but it was picking up. They went out soliciting for feedback, to expand their market reach. Somebody said they should add AI.

My friend spent a year adding a genAI-powered chatbot. It was fine, a small veneer of custom prompt over chatGPT chat. Turns out nobody actually wanted it. It was meant as a suggestion in anticipation of customer requests. The small cadre of customers wished for a way to remove the chatbox entirely. He doesn’t have much to show for the year of time and money he invested in the feature.

“Companies Are Pouring Billions Into A.I., It Has Yet to Pay Off” wrote the New York Times a week ago. “That’s a game changer”, they quote a person then quickly adds that the shift will take at least five years. Some game-changer, huh? Nobody in the article seems to know where the returns will come from. The article brings up the JPMorgan Chase claim that their employees are saving four hours weekly “on basic business tasks” thanks to their business chatbot. It’s not made clear what brought about the 10% productivity gain.

Companies seem to be FOMOing their way into genAI. They understand that if they don’t jump into the Gen AI hype cycle, they’ll be left behind. That their competitors will, all of a sudden, gobble up the market, powered by AI. An executive I know describes genAI as “Rocket Boosters for our Business”. Somebody else said it was a “momentous point in history”. There’s no clear reason to believe if any of that’s true.

Companies like the ones quoted have been sold the need to ‘invest’ in GenAI. They have seen the demos and fantastic productivity claims made by genAI vendors. The seemingly valuable outcomes are never specifically quantified, and never qualified. Where exactly will the proclaimed growth in revenue / savings come from? And will the associated development costs and maintenance expenses be covered? You don’t hear answers to those questions much. The numbers are often based on fantasy ‘agents saved’ numbers. The relationship between those numbers and reality is the same as that between :rocket-ship::rocket-ship::rocket-ship: emojis and …rocket…ships…

After wading through hundreds of such articles I have identified two classes of ‘real’ benefits. First: automating customer support. Second: a better search engine to search within existing knowledge bases of the company. And neither case justifies the often ridiculous price tags that come with those systems.

I haven’t met a single person here in Seattle whose non-Ai company has had a positive RoI (return on Investment) with GenAI product integration¹. I have asked around. I genuinely, really want to know where the gains will come from. I’m open – no, I’m eager – to find a single counter-example. I find it likely that besides in very few industries with low-hanging fruits that can leverage genAI/LLM’s to improve their products, the juice is not there. No positive RoI, no significant expense reduction, no serious revenue increase.

We’ve mistaken the forest for the trees. Executives have tricked themselves by misunderstanding the jagged frontier of generative AI. Ethan Mollick writes about it here . Generative AI’s capabilities are very ‘jagged’ – the models are quite good at certain tasks, and very poor at others. The distinction is difficult to understand clearly. However, the genAi salespeople and the media hype-sters have sold us on the furthest edge of the frontier. They’ve conveniently ‘forgotten’ to let on that the jagged frontier goes allll the way back too. They don’t tell their customers that most of their tasks lie in the less-exciting backwaters. For those, generative Ai models are no good. When I say ‘no good’, I don’t mean they’ll never be able to accomplish a given task. What I mean is, they’re not at all worth the investment. It doesn’t make sense to have each internal documentation search call cost $10 per API call. I’m not making the number up, that is the order of what a system like that can cost with initial development expenses and TCO. And that’s one of the simpler, more successful use cases of LLM’s and genAI.

I’m disappointed that the genAI hammer is used to nail down every problem. It’s like using a bulldozer to hammer nails into the wall, because the bulldozer salesperson was really persuasive! Unfortunately that’s the approach I see a teams taking. LLM’s are often wrong tools for the job, and will never get a positive RoI compared to the simpler alternatives. I wish folks looked to solving problems using any technology available. If that were LLM’s or genAI, so be it, but often it turns out not to be.

Upper management have been sold genAI as the general solution to all their problems. Anil Dash² calls it the “Gen AI Virus” , where vendors capture the management mindshare against the interest of their organizations. No wonder then, that 95% of AI projects turn out to be value negative. I so want to talk to the 5% success, and verify if they’re properly accounting all the costs. I want to believe! Give me a reason! Resource-hungry, expensive, processing is being deployed in situations where it is not needed at all. Such solutions are costlier, and inefficient compared to alternatives, and never make back the original investment. Not just that, they can end up as resource sinks, and associated opportunity costs can be massive³.

If you’re looking to go on a Gen AI journey integrating random LLM-based solutions into your product, understand that you will need to make it value positive. One day the magical aura will fade and the bills will add up. You don’t want to be on the chopping block because your costs have ballooned up with no justification! Be on the lookout, find out the Arr-Ohh-Eye, on your Gen-Ayy-Eye before you go to far.

Also READ: AI Mass Delusion Event, at The Atlantic.