Editor’s note: This is the second of two reports on the challenge of measuring return on investment from generative artificial intelligence. In part one, CFO Dive described how financial executives are plowing billions of dollars into generative artificial intelligence without a solid estimate of the potential gains.
To institutional investors, “speculation” is a four-letter word — unless they happen to make a killing. For financial executives, acting on speculation is often a surefire trigger to job loss.
Yet CFOs across several industries are piling billions of dollars into generative artificial intelligence based on fuzzy forecasts of the likely payoff.
“It’s a noisy estimate,” said Daniel Rock, co-founder of AI consultancy Workhelix, referring to common projections of the gains from generative AI. “It doesn’t mean trying is not worth it.”
Generative AI has helped CFOs finely tune customer service, refine forecasting, speed software updates and upgrade other tasks.
Yet as AI brings “the industrialization of knowledge production,” financial executives face a challenge accurately forecasting the return on investment from new, powerful efforts to squeeze more value from data, according to Laura Veldkamp, a finance professor at Columbia University’s Business School.
The challenge often breaks the usual yardsticks for ROI: Data by its nature is harder to observe and price than assets that undergird the industrial era economy such as buildings and employees, Veldkamp said at a symposium on productivity held by the New York Federal Reserve in February.
“There’s a lot of things in this economy in general that we are going to have a hard time measuring and that we just don’t know enough about yet,” Prasanna Tambe, an associate professor at the University of Pennsylvania’s Wharton School, said at the symposium. “Measurement remains a challenge.”
Despite ambiguity, the stampede into generative AI has swept up investors, financial executives and information technology companies of all sizes, CFOs and AI experts said.
“Speculative frenzies are part of technology, and so they are not something to be afraid of,” David Cahn, a partner at Sequoia Capital, said in a June research note. He sees a $600 billion gap between what companies are spending on AI-related infrastructure and the revenue needed to justify such outlays.
“Those who remain level-headed through this moment have the chance to build extremely important companies,” Cahn said. “But we need to make sure not to believe in the delusion that has now spread from Silicon Valley to the rest of the country, and indeed the world. That delusion says that we’re all going to get rich quick, because AGI [artificial general intelligence] is coming tomorrow.”
Financial executives willing to chance an outlay in generative AI, the CFOs and AI experts said, can follow seven tips to cut through the forecasting haze and seize the potential payoffs:
1. Approach ROI Flexibly
A rush into any “new, new thing” cries out for CFO skepticism. Yet when a potentially terrain-shifting technology emerges, top financial executives would do well to consider more flexibility in gauging ROI.
“As a CFO, the first thing you’ll say to someone coming to you for money is, ‘What’s the ROI?’” Glenn Hopper, CFO at Eventus Advisory Group, said in an interview. “For something like AI, it’s very difficult to determine.”
Greater efficiency is often one of the most immediate payoffs, he said. AI can trim by as much as three days the time between delivery of a good or service and payment, or the day-to-sales outstanding, he said. It can also streamline just-in-time inventory control and avert errors.
“It’s hard without any prior knowledge to evaluate the potential return on investment,” EY-Parthenon Chief Economist Gregory Daco said in an interview. Identifying with precision the potential payoffs for each company sweeps away some of the fuzziness on potential gains.
“We work on a case-by-case basis with a number of clients to say, ‘OK, in your specific sector, for your company with X market position and an X total addressable market, we believe that AI could have this this type of return on investment if applied across these functions,” Daco said. “It’s really a case-by-case approach because every industry and company is different.”
Many of the companies that succeed with generative AI will embrace risk and experiment despite uncertainty over ROI, Rock said in an interview.
“If you get failure, that doesn’t necessarily mean you made a bad choice,” Rock said. “You can cut it off and re-route it to something else.” Rock founded Workhelix with Erik Brynjolfsson and Andy McAfee.
C-suite executives this year have shown flexibility in their expectations for ROI from generative AI, KPMG found in a June survey.
During the first quarter, 51% of business leaders expected the biggest gains in the next 12 months to come from higher productivity, with 47% expecting higher revenue as the No. 1 benefit, KPMG said.
The rankings flipped in Q2, with 52% of 100 respondents ranking higher revenue as the leading improvement, while greater productivity fell to 40%, according to KPMG.
Business leaders expect the biggest gain from generative AI to come from higher revenue rather than higher productivity
“How are you measuring your organization’s return on investment related to generative artificial intelligence?” Initial response from Q1 2024 compared with Q2.
2. Focus on longer term gains
Getting generative AI up and running often demands sizable up-front investment that could otherwise go to other operations with more certain, immediate returns, Rock said.
“There’s opportunity costs to build up something intangible, like a new way of doing things, a new organizational structure or new workflows, and it seems like you’re putting a lot in to get nothing out,” he said. “Later, there’s a sense that ‘Hey, we’re getting a lot more productive’ — you start to get tangible benefits.”
Paradoxically, CFOs most knowledgeable about AI — its capabilities, limitations and costs — tend to expect the benefits over three to five years, Daco said. Their peers with a weak grasp on the technology “tend to believe the benefits will be more immediate — in two years.”
3. Create a ‘wholesale’ strategic plan
Before considering potential return on investment, a CFO should help create a strategic plan for streamlining operations and increasing growth by weaving generative AI into the organization, Daco said. Bolting the technology onto existing operations often yields meager returns.
“It’s not just about saying, ‘Well, we can automate or augment certain functions,’” he said. “It’s about having a wholesale perspective on how to integrate AI into the fabric of an organization rather than just simple add ons.”
As with any strategic plan, goal setting from the start is essential, according to Adriana Carpenter, CFO at Emburse, a provider of expense management software. “It’s always best to start with the question: ‘What is the outcome that I’m trying to achieve?’” she said.
While creating a broad strategy from a high-elevation viewpoint, CFOs should also help ensure that employees, all the way down to front-line staff, will make the most of generative AI, the CFOs and AI experts said.
“You need to develop a corporate culture that is AI savvy, and think about how to refine your workforce in a way that allows you to put into place a top-down, strategy perspective with a bottom-up, operational perspective,” Daco said.
4. Prioritize your AI projects
A CFO could apply generative AI to most company functions but should resist that temptation and prioritize instead, the financial executives and AI experts said.
“You probably want to stack-rank the different opportunities that are there because there’s lots of them and you definitely have limited resources,” Rock said.
Along with alignment to strategy, a generative AI project should draw on existing staff, focus on an operation that they will find compelling and offer the prospect of a quick success, he said.
Before launching a proof of concept, “pick something that you know will be successful and will change minds,” Rock said. “When you’ve demonstrated ROI, then you can say, ‘Cool, now we can move on.’”
Some early adopters of generative AI, although confident in eventually achieving company-wide gains, focused first on picking a surefire project rather than risk a failure that would sour employees on the technology, Rock said.
“You have to prioritize the business outcome you want to achieve,” Carpenter said.
Generative AI has proven especially potent in boosting revenue by analyzing customer data and monetizing it, Carpenter said. It also can help identify where a company is underpricing compared with the competition or how it can expand its customer base.
5. Befriend your CTO, CIO
As the holder of the purse strings, the CFO may have a tense relationship with the chief information officer or chief technology officer, who may not always get funding for their top projects, the AI experts and financial experts said. A CFO should set a friendly, collaborative tone.
“The CFO and CIO — in general — their areas are historically considered as cost centers, so they’re both fighting with each other,” Hopper said. “It’s important to collaborate together on this to gain efficiencies and really understand that AI is a strategic capability that’s going to strengthen the financial performance and operational excellence of the company — if they work together on this, everybody wins.”
The CFO also needs a deeper understanding of technology than in previous decades, the AI experts and CFOs said.
Previously, the CFO would have largely focused on how AI and automation would help the controllership, the back office and other tasks within the finance function. Now a CFO needs to cast a broader net and understand the value of generative AI across the entire company.
“I’m not saying you have to become a data scientist or developer, but you need to understand your business, your industry so you have this breadth of knowledge,” Hopper said. “If you don’t know how the information is created, how will you be able to correct it?”
A tech-savvy CFO will also sooner identify high-return innovations in generative AI.
“There’s all sorts of new breakthroughs occurring — every week there’s some new fascinating thing,” Rock said. “You need to stay on top of new developments and build your own detector for the impossibly high-value innovation for your company.”
6. ‘Define’ the data
As part of accurate ROI measurement, the CFO needs to ensure a precise, company-wide definition of the data fed into generative AI, the AI experts and CFOs said. Otherwise, different parts of the company may use the same data to make inconsistent conclusions, undermining the technology’s value.
“In the early days you ended up with these data silos because everybody was doing their own citizen development and building their own data fiefdoms,” Hopper said. “If you’re only taking a slice of the pie at a time, you’re not going to get maximum value. The way you’ll get maximum value is through a shared data warehouse and using it across the company.”
CFOs should keep in mind that the costs of launching a generative AI application — including creating a “data dictionary” and detailed governance of information as part of “responsible AI” — may exceed estimates, Carpenter said.
7. Bring employees along with you
CFOs seeking to achieve a high ROI will err by deploying generative AI primarily to cut payroll, the AI experts and CFOs said. Instead, the technology offers a potential tool for streamlining operations and lifting employee job satisfaction, they said.
“It’s a massive mistake to view this as a cost-saving technology in terms of workforce rather than an augmenter that enables people to get a ton of work done and focus on things that are genuinely hard to do,” Rock said.
Large-language models such as ChatGPT will imperil jobs for workers in dozens of professions, including accountants, auditors, financial quantitative analysts, blockchain engineers, interpreters, mathematicians and journalists, according to a study by researchers at the University of Pennsylvania and OpenAI.
The technology over time will streamline at least 10% of the tasks performed by 80% of workers, and half of the tasks done by 19% of workers, the researchers said.
To dispel worker anxiety, CFOs should “cultivate an AI-savvy culture with a desire to move things forward,” Daco said. They should ensure that employees are “on board with the transformation and that fears of job loss are alleviated.”
Mastering the ability to precisely gauge ROI from generative AI will not happen in a matter of months, the financial executives and AI experts said. To succeed, CFOs will need a balanced mix of resolve and patience.
“There are a lot of learnings that need to occur, there’s going to be fits and starts and it will take time for each type of business in different industries to evolve and accelerate the adoption of this,” Carpenter said. “But generative AI is here to stay.”
