Rubrik CIO on GenAI’s Looming Technical Debt

The race to adopt new generative artificial intelligence (GenAI) tools and platforms is causing concerns about a future pile-up of technical debt -- the cost of favoring speed of adoption over cautious development.

Technical debt is hardly a new concept and not exclusive to artificial intelligence. A 2023 CompTIA survey found 74% of organizations said technical debt posed challenges. And 42% of those surveyed said the technical debt would cause “substantial hindrance.” Technical debt is often an invisible foe that can creep up on IT leaders as companies push to grab a competitive advantage with emerging technologies.

According to Crunchbase, more than 25% of investments in American startups went to AI-related companies in 2023. The global AI market size was about $208 billion in 2023 and expected to reach $2 trillion by 2030, according to Statista. All that hype and spending is creating demand for quick implementation and quick return on massive investment. In turn, worries of quickly mounting technical debt are growing.

Rubrik CIO Ajay Sabhlok, who also serves as the cloud data management company’s chief data officer, tells InformationWeek that the threat of massive technical debt due to explosive GenAI growth is real, but not insurmountable.

Related:Technical Debt: Strategies to Quell the Quick-Fix Quagmire

(Editor’s note: The following quotes are edited for clarity and brevity).

Can you give us an overview of AI’s contribution to technical debt?

Technical debt is not a new concept. We’re all very familiar with how it gets generated. And when we started looking at GenAI -- the combination of many different models, LLMs (large language models) that are available to the market, along with other platforms for development -- a variety of things started happening. For the sake of acceleration agility, you don’t want to start establishing standards from day one. A lot of us are in experimentation mode, we are learning, we are creating. And some of the use cases we are putting out rapidly because you want to increase acceleration for time-to-market. So, it is likely that we will develop some tech debt, because there are a variety of teams that are pursuing different solutions, platforms, and offerings from different vendors. And in that process, there’s not a whole lot of standardization going on within the company. I think it’s an experience very similar to when we started adopting SaaS in a big way. And we had various strategies we had to deploy to try to address that. It’s going to be very similar here.

Does GenAI’s technical debt go beyond what you experienced with other sorts of tech debt, considering how this could impact automation not just in the back office, but throughout the organization? How do you mitigate that impact?

Related:How Technical Debt Hampers Modernization Efforts for Organizations

GenAI is very broad. So, there are tools and technologies that are homegrown, SaaS vendors are expanding their own offerings in the cloud, and probably on premises -- it’s available for people to consume and experiment. So, the question that comes to mind is: What do I do about it? This is a case of, “Hey, there’s a leak in the boat, and what are you going to do about it? Are you going to let things get drowned? Or are you going to make sure that there is an equal amount of water that leaves the boat?” So, you have to apply that thinking to your annual plan. Typically, I’ll say that there’s going to be a percentage of resources, budget, and effort I’m going to put into reducing tech debt … And that’s where you start competing with other business initiatives. You will have a bunch of business stakeholders that might look at that as something that should just be kicked down the road because they want to use that funding for something else. That’s where, I believe, educating a lot of my business leaders on what that does to the organization. When I don’t address that tech debt, on a regular basis, production SLAs (service level agreements) start to deteriorate. So, tech debt needs to be treated in a more strategic manner.

Related:$10 Million AI on a $2 Budget

The technical debt accruing with generative AI, is it going to be magnitudes more than other technologies we’ve seen in the past? Or is it just the same thing with a new flavor?

We probably got a little overwhelmed because a lot of things were thrown at us (with GenAI adoption), but then we started to realize that it’s not that we’re adopting everything all at once. We are actually being a bit cautious about what we adopt as a long-term platform. So, we’re doing our experimentation, maybe we put out some solutions into production, just to test drive … but there will be a shakeout. There’s going to be some consolidation and some standardization across the board. So, the first couple of years are going to be rocky very everybody. But that doesn’t scare us, because we’re going to put a more robust governance on top of this new area. We need to have a lot more debates about this internally and say, “Let’s be cautious, guys. Because this is coming from all sides.”

Is preparing for AI technical debt going to be a tough sell considering the macroeconomic environment? There’s a lot of cost cutting going on. Is it going to be hard to sell executives on the idea of investing in a future unseen problem?

We chatted with a lot of CIOs about funding … And there’s not a whole lot of new funding that’s coming in when we talk about GenAI experimentation. We’re kind of using our current resources. So, I’m looking at the existing framework of how I manage things and saying, "Can I give this task to an existing team?" Because maybe I can’t afford to build new governance teams and I can’t invest in new headcount and new resources. That resonates with a lot of CIOs that I’ve talked to … it’s something that we are baking into our existing set of resources. It’s going to be a little more intense because we are all learning a new platform and a new skill.

When do you think things will level out? What do you think the outlook will be with GenAI and technical debt?

In two years, I expect a bit of standardization. I expect some major players to emerge that will start to influence a lot of customers. With GenAI, we’ll just have to see how people gravitate … because it’s ubiquitous. I think it will become part of everything we do -- every tool, every single process and we will have to bake that into our day-to-day designs and solutions … You could put out some really amazing solutions with very little effort because of how powerful these new technologies are. In two years, I really think that will start to stabilize and we will see the bigger picture of what we want to do with GenAI and I think we’ll have a better handle on it. Up until then, we should approach with caution … It’s also about being aware of all the mechanisms you have already have in place and making sure you leverage them to your advantage.