Discover Financial Services exec: GenAI isn’t yet ready for prime time

As generative AI (genAI) tools grow in adoption and sophistication, Fortune 500 companies are finding they can rely more on data to drive decisions and improve operations.

For more than two years, Discover Financial Services — one of those Fortune 500 companies — has explored how it can use genAI to improve quality and create efficiencies. But in a heavily regulated industry like financial services, protecting sensitive customer data always comes first, which has hampered the arrival of more advanced use cases for genAI.

Coupled with the arrival of agentic AI — a more autonomous version of genAI that can make independent decisions with far less human oversight — the combination can be the thing of nightmares.

Computerworld spoke with Keith Toney, president of Credit and Decision Management at Discover Financial Services (DFS) and a member of the company’s executive committee, about what the company has been doing and what it’s learned. Toney, who also serves as co-president for all of Consumer Banking, has helped lead efforts to bring about enterprise-wide adoption of advanced decision science.

After starting at Discover Financial Services in 2019 as the Chief Data Officer responsible for the enterprise technology organization, he joined the DFS Executive Committee the following year. With more than 25 years of experience in financial services, analytics, and the economics of risk pooling, Toney specializes in emerging technologies in big data, AI, machine learning, data visualization capabilities, data governance, and data security. He leads an area called decisions and analytics and reports directly to DFS’s CEO J. Michael Sheppard.

The following are excerpts from the interview with Toney.

srcset='https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?quality=50&strip=all 1237w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=300%2C291&quality=50&strip=all 300w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=768%2C745&quality=50&strip=all 768w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=1024%2C993&quality=50&strip=all 1024w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=718%2C697&quality=50&strip=all 718w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=173%2C168&quality=50&strip=all 173w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=87%2C84&quality=50&strip=all 87w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=495%2C480&quality=50&strip=all 495w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=371%2C360&quality=50&strip=all 371w, https://b2b-contenthub.com/wp-content/uploads/2025/04/Keith-Toney-Headshot-2.jpg?resize=258%2C250&quality=50&strip=all 258w' width='1024' height='993' sizes='(max-width: 1024px) 100vw, 1024px'>Keith Toney
Discover Financial Services

So, we’ve not spoken before, but I have spoken to Discover Financial Services about its use of AI. How have you been involved in that tech rollout? “I think you had a conversation with one of my direct reports a bit ago, Raghu Kulkarni, who is kind of the chief data scientist, but I lead this area. I have about 1,500 people in my organization, and we’re doing machine learning and analytics, data work and insight generation.

“It’s probably best to describe it as both a horizontal and a vertical, which is a little bit funny and a little bit unique. Horizontal being that we do work all across the bank, all across the value chain, from brand and marketing all the way through to collections and recovery. So, if you think about the life cycle of a consumer with … lending products or banking products — that whole value chain— that whole horizontal, I have.

“The vertical part, though, is also interesting. I hold business responsibility for things like credit, which is the primary thing that you do in lending — make underwriting or credit decision, that’s part of my responsibility. That includes all the machine learning models that make the decision on whether or not we’re going to give you a credit card or a personal loan or a home equity line of credit, or make decisions in the checking and deposit space, the credit fraud collection strategy, the digital and mobile teams and so forth. All those algorithms are all part of my responsibility.

“The principle argument for all of that is that as a direct-to-consumer digital bank, so much of what we do is run off these algorithms. So, we decided to bring that together under a common leadership structure.”

That’s exactly the challenges a lot of entities are having right now with genAI tools — integrating them with existing applications. Salesforce, for example, has infused AI into its software. And you might have your own instance of AI that you’ve created, and you want that to be able to speak to the AI in the your Salesforce platform. How did you prepare your infrastructure, your data, for the introduction of AI? “We’re not in the business of building a new big LLM. We’re, going to buy that from a vendor. But we’re tailoring them. We’re refining them. There’s a process of fine-tuning these models to have it interact within the data that’s within the company, and that data may not be organized for that purpose. When we were capturing it, we weren’t really thinking about tagging it and structuring it and organizing it in the way that makes it as accessible for these kind of tools to work.

“So, we’re piloting several things. We do a lot of work alongside [our CIO] in making custom-fit models, working in certain kind of use cases. But we’re also using Salesforce and Microsoft Copilot. We’re a Microsoft shop.

“We’re rolling out Copilot, and we’ve had to go through a lot of work to make sure we’ve been thoughtful about data classification, restricted data structures and so forth, so that we’re making it available to the larger part of the organization. At the same time, we’re ensuring we’re still protecting restricted and secure data.”

Let’s talk about data classification, data structures, security, and setting up guardrails, because we know you can’t just set it and forget it with AI. What were some of the guardrails? What were some of the precautions you took? “So, we have a data classification scheme that has five or six levels to it. Basically, we’ve been rolling that out as an enterprise program for quite some time.

“The most loose classification would be data that can be made available to the public. And then you have kind of internal and confidential data. Then you have restricted, and then restricted with supervisory information. We’re a highly regulated bank, and so we’re having a back and forth with a regulator all the time — someone in the Federal Reserve or the Office of the Comptroller, or FDIC. Those are our regulators. That’s, that’s highly classified, restricted data. So think about like that hierarchy — getting that classification structure rolled out so that all the all the data elements are essentially classified. And then looking at the directory structure, where data sits within the enterprise.

“You’re probably familiar with SharePoint or OneDrive. These are Microsoft applications. But there’s others, whether you’re a Google shop or whatever. Data lives in these directory structures. Some of it is very structured data, some of it is unstructured data, like audio. We run a big call center, we record those calls. Those are audio files. That’s obviously unstructured data, but the file itself is cataloged and structured and gets tagged. And that contains sensitive, restricted data.

“A lot of times we’re talking about [personally identifiable information]. So there’s a kind of a matrix of classifications and labels that lead to handling standards and what we want to make available inside of the AI environment. For example, if it’s a confidential document in one area, I don’t want Microsoft Copilot, which is kind of navigating across all of my infrastructure, to be able to go pick that up and make it available to someone who shouldn’t have access to that that document. It’s those kinds of challenges we’re facing with AI.”

Have you gotten to the point where you feel comfortable rolling Copilot to all of your employees, where you still feel that that your data would be protected. “We are in the process of kind of rolling that out, and we’ve learned a lot. I think probably what you’re hearing from many folks in the industry, across all the industries, is [genAI] implementations have slowed, and it’s largely because of issues like this [navigating security, privacy and data strutures].

“And there are some areas where we’re very comfortable. We have the data in a container that we feel is super serious. There are some areas that we’ve left cordoned off, and we’re not exposing that to the application until we get further confidence on the classification scheme that there’s not data left to be discovered in those spaces and so forth. So as you can imagine, it’s just a very rational process of progressively exposing it and making it available.

“So, some areas were much more comfortable. Some areas we’re still working on it.”

What are some of the areas, if you can talk about them, that you’re less comfortable with? “Well, it’s a natural fallout of the description I gave you. You know all your  documents in legal and compliance. We’re not necessarily mixing those with the broader implementation of Copilot, as an example. So, how you think about sensitive and highly confidential or restricted material, and then how that’s made available in the platform. We’re not a big Salesforce shop, but I think you have kind of the same issue in any platform that you want to make available. Do you have command of the restricted data, confidential data, and then things that are just like largely internal, or things that you would consider public?

Where do you see the return on investment with genAI? “The top track, which I think is naive, is just around job elimination — the idea that AI is going to take over and replace jobs. I don’t feel like that will be the out-of-the-gate impact. There’s so much backlog of ideas and work and development in our business, and … in other kinds of businesses, as well.

“It’s going to be some time before we get AI to the place where we’re making our existing employees so much more efficient. We’re attacking the backlog right now; adding more features and capabilities and other things to our products and services before we go to the place where we’re just looking to cut costs and reduce jobs. I just think that’s the reality of it.

“I think efficiencies like that will come sooner or later, but you know, so much of the work that we do is technology, and technical. We’re using the tools to make developers more efficient and more effective in their job. But again, there’s such a backlog of work to do that that’s why I tend to challenge a little bit of the doomsday narrative — at least in the short term. For the next two or three years, that’s how I think about it.”

It sounds like you think that genAI is going to help developers. Have you seen that already? And do you see at least replacing some of the lower-level developer or even mid-level developer jobs? “If I squint, I can see it. It is starting to happen. For example, if you write SQL, so just a query language against the database, it’s very easy now to have Copilot help you write that code, whereas you used to have to learn it in a different way, have a different depth of knowledge. Now, you’re able to execute code much faster, or get to working code much faster. And so you play that forward. It does seem like there’s some point in the future where that’s going to create less demand [for developers]. But in my opinion, it’s also accelerating those entry-level employees up the learning curve faster.

“Job [up]-skilling is a really important piece of the conversation. I just think that some of this will naturally reach a different equilibrium in terms of the work. But again, there are so many interesting things to do improve the tech stack. Plus, obviously, we’ve got to get through the change management curve. A lot of these systems used to be somewhat more manual, or maybe based on machine learning. We’re now wanting to move them to a more of an artificial intelligence platform.

“That’s not easy to do. These systems are complex. You have to be very thoughtful. And then, you have to consider that we’re in a highly regulated business. We have to be just super mindful of how it impacts consumer, and make sure that, like, we’re not putting in place something that will, you know, kind of miscalculate the kind of the outcome that you’re seeking.”

Have you deployed any AI agents? “We have not. We still operate fully with a of human in the [AI] loop.”

A big topic now is agentic AI — the combination of the large language models and process automation. So, the notion of having agents that can act based autonomously speaks to a great deal of potential efficiencies. Would that be something that interests you? “Yes. You can still have human supervision, but at a far lesser degree. The applications that we’re piloting and the things that we have in flight are mainly working with our human agents in the call center. These applications are largely acting as assistants that our human agents are still in command of and ultimately, the information that gets shared back with the customer, or that kind of gets shared into the process.

“The AI implementations are creating acceleration of finding customer complaints or other things in the data. But we’re not yet comfortable with kind of a true agentic implementation. There’s still a lot of still a lot of tethering needed.”

Tell me about your data visualization efforts. “Data visualization for us is primarily dashboarding and reporting around business performance, or consumer behavior, shopping behavior, and those kinds of things.

“We’re looking for ways to see patterns in data that can help us and through visualization, take different actions. So we’re studying multivariate structures around fraud, for example. And fraudsters are constantly trying to circumvent the system to their advantage and compromise either the consumer or the system itself. So, we do a bit of data visualization and pattern recognition as part of an early warning system. If we see something that’s out of pattern, we ask if that’s a bad actor or a ring of bad actors that are kind of causing us some kind of a fraudulent issue.”

So, AI is able to assist with you in that? “We’re using machine learning. So, when you say AI, if you mean large language models and generative AI, not so much — yet. We do use machine learning and other kind of graph database technologies and things that I put under the broad banner of AI — generative AI being a piece of that machine learning.

“We’re exploring [genAI] for that. We’re trying to leverage the compute power that’s behind some of the generative AI and the large language work to see if we can use that to determine new patterns. One area that we’ve done a lot of work in that’s under the bigger banner up front is money laundering. We have a big responsibility to study all the transaction flow.

“We need to know if there’s bad actors out there that [are] trying to launder money through the system. So we’ve been looking at AI, not only for pattern recognition in terms of anti-money laundering, but also in the support of the agents.

“When we see a pattern, a lot of it then becomes research and documentation of those kind of suspected activity reports. That’s one case where we really think there’s some interesting applications of the generative AI.”

Are there any other areas that you see a prominent use case for generative AI? “The biggest forefront for us on generative AI is really in the context of the call center experience and the customer experience and integrating that with the digital app and the website. Discover wins JD Power Awards for customer experience. And we think generative AI and those applications are going to be are going to be pretty powerful. I’ll tell you, though, what’s interesting is, from my perspective, there’s some areas where we don’t see it being useful in, like underwriting, credit and underwriting, which is under my responsibility.

“We’re not seeing an application because of AI’s [problem with] hallucination. These are probabilistic models. They’re not deterministic. Therefore, you can ask it the same question twice, you can get a different answer. In a world where I am, if I’m underwriting to decide whether I’m going to give you a credit card, I can’t have it be — if I ask it twice — ‘Oh, I will give you one. Oh, I won’t give you one.’

“There’s some areas that clearly, depending on the sensitivity of the decision that you’re making and the implications of it, that you wouldn’t want to use generative AI. That credit underwriting one is a good example. Other things around like liquidity and trading and so forth, are other examples.

“You need to understand how the [AI] models can ‘hallucinate’ and create two different answers if you ask it the same question twice. And that’s also why we think the human in the loop will stay important for some time in various scenarios.”

How have you approached training your employees? What percentage of your workers have been trained on AI? “You can think of it as like concentric circles. We have a relatively small group of researchers who are super deep in generative AI, machine learning and all of that. And then we have a set of users that are in our product organization, and they’re thinking about how to leverage it. And then we have the general staff.

“At the broadest level, training right now is on the usage of Copilot and some of the general tools associated with it. And then we’re training on data protection and those kinds of things. So we’re focused more on general consumption and the actual usage of the models and the development of applications, and they’re in there running the pilots. We have folks that are that are really deep in the technical mathematics of it, all the way out to folks that are general users. So we’ve had to kind of think about it in different layers.”

Have you trained a large percentage of your users, or are you still in the process of figuring out who needs that training? “We’re still in the process of figuring that out. There’s an Advanced Analytic Resource Center or AARC. This is the thing that we did in downtown Chicago, where we’re taking early-in-career, out-of-college new grads and bringing them into the company through a structured training program around the tools and technologies we need, and then accelerating them through like a two-year development cycle. Then we place them in the business. That’s becoming one of the areas where we’re incubating knowledge around machine learning and generative AI.”

Have you discovered where the greatest ROI in genAI may be? “In some sense it’s the classic Gartner Hype Cycle. We’re in the trough of disillusionment. I think we’ve been pretty sober about the implications of it, and we’ve been very deliberate. There’s certainly some companies that have really gone all in, but they’re in less regulated industries; they have the ability to do that in a bit more of a unconstrained way.

“Given the nature of banking, and especially direct-to-consumer credit card lending and so forth, we just had to be really deliberate and thoughtful.”

Where do you see the big areas that genAI and agentic AI will be of most use in the future — say two, three or five years down the road? “It’s a call center, and then the other parts of the customer experience. So, the blending of the mobile app, the mobile experience, the web experience, and then the call center experience.

“I don’t think the call center experience is going to completely go away. Financial products are just too complicated, and people just need to talk to somebody to engage with that. So, first, I think it’s call center. Second is the full customer experience. And third, it’s the brand and marketing space.

“Discover is a big brand, and we spend lots of money on marketing, advertising and so forth. And the content around brand and media is ripe for improvemnt with generated data.”