The use of generative AI seems to be helping employees be more productive, in banks and other workplaces. But so far, the results are far better for young people and novices than for skilled workers, who can even be negatively impacted by the technology.
Use cases for generative AI — large language models that can create new content based on prompts — abound in banking. Citizens Bank is creating an AI copilot to help call center reps answer questions. Citi is rolling out GitHub Copilot to all its developers. Ally Financial is using generative AI to provide post-call summary recordings in contact centers. JPMorgan Chase is providing large language models to all its employees to generate drafts of emails and reports; it also uses gen AI to understand and detect phishing attacks.
“It’s like we just released Excel and it’s 1980,” said Michael Abbott, global banking lead at Accenture, in an interview. “You’ve got this unbelievably new tool, but nobody knows how to use it. It’s going to change the way we do work, but people are still figuring out what it can do.”
Where gen AI is making bank employees more efficient
Generative AI is bringing efficiencies to banks “in some pockets,” according to Alexandra Mousavizadeh, CEO of Evident, which released its AI Index on Thursday. “There definitely are areas where it’s really working. It tends to be when it’s quite a small proof of concept – let’s try and fix these model driven mistakes in our trading platform. And they use gen AI to rethink the process and identify them.”
In wealth management, some banks are seeing real returns now, she said in an interview. “It depends very much on, is the bank making the effort to do the training? Because you can create the tools and give them to the analysts, but actually they’re a pain to operate. People just tend to not use them.”
One place banks are starting to see time savings from the use of generative AI is in software development (like Citi’s use of Github Copilot). Banks are seeing 30% to 50% productivity improvements in this area, according to Alenka Grealish, principal analyst at Celent, who spoke in an American Banker podcast that will air October 22.
To get the full value of giving software developers generative AI, banks have to build a “virtuous cycle” around it, said Xavier Lhuer, McKinsey partner, in an interview.
“If you look at the way software developers spend their time, they typically only spend 30% to 40% of their time in writing the code, interacting with the program manager to figure out what to build,” Lhuer said. More than 60% of their time is spent in deploying the software and getting governance approvals and infrastructure provisions. Removing that bureaucracy could help make developers more productive.
Good for newbies, bad for experienced staff
In one use case for generative AI in banks – helping call center agents do their jobs – a study conducted by researchers at Stanford University and MIT found that the technology helps only low-skilled workers.
The researchers studied three million conversations between customers and 5,179 customer support agents at a large software company. Some of the agents used an AI chatbot based on OpenAI’s GPT-3 that generated suggestions for how agents should respond to customers as well as links to the company’s internal documentation for technical issues.
On average, access to the tool increased productivity, as measured by the number of chats a worker can resolve per hour, by 14%. Among novice and low-skilled workers, there was a 34% improvement.
But among the highest-skilled workers, the researchers saw no difference in call handle time and small but statistically significant decreases in resolution rates and customer satisfaction.
“These results are consistent with the idea that generative AI tools may function by exposing lower-skill workers to the best practices of higher-skill workers,” the report states. “Lower-skill workers benefit because AI assistance provides them with new solutions, whereas the best performers may see little benefit from being exposed to their own best practices.”
The negative effect on productivity among the experienced agents “suggests that AI recommendations may distract top performers, or lead them to choose the faster option (following suggestions) rather than taking the time to come up with their own responses.”
Abbott has seen this dynamic in banks he works with.
“What we find with generative AI is that you rarely ever make the best better, but you make the low-end and middle better, and therefore you shift the whole curve,” he said. In a test of knowledge management at a call center, for instance, Accenture found that novice users became more efficient, but experts got nothing out of it, “because they already knew where all the problems were and they knew how to solve them already.”
Mousavizadeh has also seen this happen among software developers using gen AI.
“What I hear is that the very experienced coders get a little frustrated with it,” she said. “They’re like, it’s easier if I just do it myself.” And among very inexperienced developers, “it doesn’t really help because they can’t spot the mistakes” the generative AI model makes, she said. Among mid-level programmers it’s useful, she said.
More darkly, the MIT/Stanford study also found that training models on the work of experienced agents and feeding the outcomes to novices takes advantage of the skilled workers.
“Top workers are generally not paid for their contributions to the training data that AI systems use to capture and disseminate their skills,” the report noted. “Yet, without these contributions, AI systems may be less effective in learning to resolve new problems. Our work therefore raises questions about how workers should be compensated for the data they provide to AI systems.”
The study also called out an interesting positive effect of generative AI on call center workers: It helped them sound more professional, which led to customers treating them better.
The report noted that call center agents are typically treated rudely by irritated customers. But using generative AI helped agents communicate more effectively, decreasing the likelihood that they would be perceived as “mechanical or inauthentic.” Customer sentiment improved and fewer customers questioned the competence of agents by requesting to speak to a supervisor.
Using generative AI to summarize calls in contact centers can shave 15% to 20% off average call handle time, according to Abbott.
Similar time savings could be reached in mortgage underwriting, he said.
“You can take the Fannie and Freddie mortgage underwriting policies and tell a large language model to act like a loan underwriter and look for all the red flags against the policies that might be in a loan,” he said. “It’ll take in all the documentation from the loan, the policy, and start looking for those red flags. There’s still a human in the loop, because you have to be assured from a model risk management perspective that you’re doing the right thing.”
In cases like this, where generative AI is taking on repetitive, boring tasks, banks could get productivity gains of around 20%, Abbott estimates.
Generative AI doesn’t make sense for actual loan decisions, Abbott pointed out. “Most banks have incorporated all the data they legally can incorporate into an underwriting process,” he said. “So adding gen AI, which is going to give you random answers, is not going to add anything to it.”
Other places where gen AI can achieve productivity improvements, in Abbott’s view, are in creating credit memos, in marketing production processes, in risk and controls, and in data mapping.
Along the way, generative AI can improve job satisfaction, Grealish said.
“If an employee can do something more interesting – help that consumer with a more challenging problem better and faster – job satisfaction is certainly going to go up,” she said. “When you don’t have to do routine, dull parts of a job, people are going to be much more interested in their job and be able to add value. And so the customer in the end is a winner.”
