Nearly a year after BNY rolled out its generative AI tool Eliza, the bank is preparing to upgrade its capabilities. The 2.0 version, expected to launch next year, will provide more detailed responses based on more complex, humanlike thought processes.
“Eliza 2.0 is a lot more about reasoning: Instead of just giving an answer, it’s about taking a step back, thinking through various options,” said Sarthak Pattanaik, head of BNY’s AI hub.
Eliza 2.0 comes roughly a year after the bank’s rollout of the technology. About a fourth of the bank’s 53,400 employees are using the tool, he said.
Planning began in August 2023, the result of the work of the bank’s AI hub, a team that pulls together roughly 200 employees across the company. The bank strives to grow adoption through ongoing training, with virtual and in-person forums that offer opportunities to explore use cases and share best practices, said Pattanaik.
“The platform was built at breakneck speed,” said Pattanaik. “We are going through a platform operating model, and getting a lot of people from various groups to understand the art of the possible,” he said.
The upgrade of BNY’s Eliza comes as other large banks and financial firms deploy generative AI tools to deliver efficiencies: JPMorgan Chase, Goldman Sachs and Morgan Stanley rolled out generative AI tools this year. BNY declined to comment on the financial investment in its AI hub, but the bank spent $3.8 billion on technology in 2023 — an amount that includes the AI hub, according to a bank spokesperson.
Use cases
Eliza is built using on-prem Nvidia GPUs and cloud infrastructure from Microsoft Azure and Google Cloud. It taps commercial large language models — including OpenAI’s GPT-4 and Google’s Gemini — and open source LLMs, including Meta’s Llama. It’s used by a range of employees, including technical team members along with client-facing and operations staff. By the end of the year, the bank will likely have more than 40 AI solutions in production, a bank spokesperson said.
Use cases run the gamut, from general to niche: A member of the legal team might use Eliza to compare documents and ask questions; a senior executive may use it to get draft talking points on a particular topic; or a technical team member might ask Eliza to translate a snippet of structured query language, or SQL, into natural language.
Eliza also includes tools to help employees do their jobs more efficiently, including, for example, a tool that analyzes funds, flags deviations and helps accountants address them, according to the bank.
Onboarding and training
To use Eliza, employees are trained on the responsible use of AI, appropriate data usage and the importance of keeping a human in the loop, according to the bank. Employees need to take a test before they can gain access.
The bank also brings together employees virtually and in-person to share insights on Eliza. An Eliza community page helps employees share information on the strengths and weaknesses of various models. Every week, employees meet to go over improvements to the platform and present use case examples. The bank also holds hackathons and “promptathons” to help employees experiment in a safe environment, said Pattanaik.
Guardrails
The bank has embedded safeguards around information security, third-party governance, vulnerability management, identity and access management, and data loss prevention, Pattanaik said.
At the model level, the bank manages risk by applying a set of key principles, including fairness (screening for bias); ethical usage; transparency (clarity on how models generate answers, including sources of information); and data privacy, he noted.
Before any use case is put into production, the bank employs an official governance process to design, build and operate it. Human-in-the-loop and other oversight mechanisms (including “challenge agents”) are also used, according to Pattanaik.
“Human in the loop is a key metric for us wherever possible,” he said. In addition, challenge agents, which could be entering the same prompt into a different large language model, can help ensure accuracy of the output.
Results
The objective of a tool like Eliza is not to eliminate jobs, but to enhance employees’ capabilities on the job, said Sharyn Jones, BNY’s global head of talent management.
“We look at it as changing the way that people do their day-to-day jobs,” she said. “If you think about what AI does from the perspective of finding opportunities for efficiency … we really want to focus on how people are spending their time.”
For each use case, success and failure evaluation criteria are set upfront, said Pattanaik. “When we go for the specific use cases, we identify productivity enhancements and execute on them,” he said. “But there is not a big number out there that we are all targeting toward.”
BNY’s efforts to roll out generative AI align with efforts of many large financial institutions. The challenge they face goes beyond technology: It’s about putting a governance structure in place to allow the tool sets to scale, said Dan Latimore, chief research officer at The Financial Revolutionist.
“It’s not just the technology, but it’s the leadership and governance around it,” said Latimore. “You’ve got to set clear goals, adjust them based on what you learn through your experiments and focus on propagating the initiatives that show the highest return.”
The success of enterprise generative AI rollouts across financial services firms will also hinge on their ability to harness their collective brain trust of intelligence on the best use cases and prompts that can be deployed across an organization, suggested Vikas Agarwal, financial services risk and regulatory leader at PwC.
For its part, BNY is betting on a longer-term payoff from its AI efforts.
“We’re using our AI Hub to collect these different use cases and then be able to deliver solutions and many AI platforms that can then be used in multiple places around the company,” CEO Robin Vince said during the company’s first-quarter earnings call in April.
“We think it’s going to be very significant over time, but it’s not a ’24 story … maybe not even ’25, but a ’26 and beyond story,” Vince said.
