Most banks experimenting with generative artificial intelligence are starting with closed-source models and gradually progressing to open-source models.
But the calculations that govern these decisions, including those around cost and security, are changing, as are other developments in this space such as a budding interest in small language models.
“We are seeing clients starting with one technology and experimenting with other models because they want to be agnostic,” said Bahvi Mehta, the global head of AI for financial services at Bain. He sees banks using generative AI models for several purposes, including frontline productivity, such as summarizing information for relationship managers; customer service, such as summoning answers to customer queries quickly; generating code; and creating content for marketing. Automating workflows in back-office operations, such as triggering notifications for compliance issues, are more nascent.
Experimentation is less about the size of the bank “and more about their innovation agenda, ambition and risk appetite,” Mehta said.
Open-source software refers to source code that is available to the general public to download, host and modify within their infrastructure. Closed source refers to proprietary models, often provided as a service, which are hosted and managed by technology companies such as Google, OpenAI and Anthropic.
Banks generally start with closed-source models because, “The fastest way to build muscle in an organization is [with a model] that already exists and doesn’t require a lot of infrastructure to manage and maintain,” said Mehta. “You are not worried about building the model itself.”
This is one plus that Imran Khan, head of TD Invent at TD Bank Group, recognizes. The Toronto, Canada-based bank, which has $2 trillion of assets in Canadian dollars, is piloting two use cases using closed-source models.
“Currently they can be easier to use and access because they are provided through third parties, and third parties give you more support,” he said.
On the other hand, open-source models afford more access to the model architecture.
“We are being flexible because the AI environment is evolving quickly,” said Khan. “We are looking to test and learn so we can better understand the benefits and drawbacks of both.”
Cost is another question. The financial burden of using a model hosted by a third party is seen as higher, because companies pay to access them on an ongoing basis. Open-source models are free to download. At the same time, open-source models require infrastructure to run and more technical expertise on staff.
“With open source you can download the model for free, but the bigger challenge is maintaining and training those models and keeping them updated,” said Mehta. “With growing expectations from customers and other stakeholders in terms of accuracy and outcome, it requires a good data science team internally to manage it.”
This is a factor for Ryan Hildebrand, chief innovation officer of the $3.2 billion-asset Bankwell Bank in New Canaan, Connecticut.
“We don’t have a huge budget to build our own models,” said Hildebrand.
With major technology companies, “this is their business,” he said. “I’ve heard even from the largest banks investing a ton in AI that though they have the best engineers in banking, no one will have the quality of talent that OpenAI, Microsoft and Google have.”
Open-source models allow for more transparency, since the organization is building on top of basic code. The ability to fine-tune closed-source models is limited.
“If you are a bank looking to use a language model as part of a use case that is sensitive or high-risk, and maximize the amount of transparency with model documentation and model testing, open source would generally speaking be more practical,” said Jey Kumarasamy, senior associate at Luminos.Law, a law firm founded by both data scientists and lawyers that focuses on AI risk.
There are questions around information security. Banks must be careful about inputting sensitive data to models hosted by a third party, and set controls to ensure their data isn’t used to train those models. Open-source models, on the other hand, allow more governance over the design and containment of sensitive data.
Still, Mehta finds that this area has come a long way in the last 18 months. Concerns about data security with closed source models are partially assuaged by vendors such as Amazon and Microsoft becoming more transparent about their security controls and adopting zero-data retention policies for clients.
There are other developments in the generative AI space beyond the open-vs.-closed source conundrum.
Kumarasamy has noticed in several sectors, including banking, a recent uptick in interest around small language models, an alternative to large language models that are typically open source and require less intensive infrastructure to run locally. They make the most sense for narrow, pre-defined use cases, he said, rather than writing an essay or analyzing a spreadsheet.
Sourish Sarkar, senior director in Moody’s solutions architecture group, has seen an interest in hybrid large language models, which let institutions deploy public and private models in parallel.
“The rapidly evolving landscape of generative AI underscores the imprudence of committing to a single strategy,” he said.
Mehta does not believe banks will commit to one model, but turn to closed source, open source and perhaps proprietary models for certain purposes.
“It will be a situation where you have all these models coexist in an organization and you pick based on purpose,” said Mehta.
Kumarasamy is finding that banks, among other companies, are building internal platforms that make it easier to swap out models so they are not tied to one.
This is the approach Ally Financial in Detroit is taking.
“We firmly believe that the future is being model-framework agnostic,” said Sathish Muthukrishnan, chief information, data and digital officer at the $192.9 billion-asset bank.
Ally has experimented with closed-source models including OpenAI’s ChatGPT series and Anthropic’s Claude, and open-source models including Google’s Flan-T5 and Meta’s Llama series. Ally.ai, a proprietary cloud-based platform that unites Ally data, generative AI models, and an orchestration layer which wraps AI models and services within a single interface, lets the bank run multiple large language models and data scientists choose which is best suited for the work they are doing — regardless of where the model is hosted or which company developed it.
Ally.ai also acts as a layer of security. For instance, the platform can strip personally identifiable information from data that passes through before using it within a model.
“The goal is not to delegate security or risk control to someone else,” said Muthukrishan. “I don’t care whether the model is open- or closed-source behind Ally.ai. I use Ally.ai to establish architecture controls, access management and usage guidelines.”
