John Roese, Dell Technologies — theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

[00:00:00] Speaker 1: - Palo Alto Studio Connecting, Silicon Valley and Wall Street. - I'm John Burr, and I hope you see here with Dave Olave, my co-host. [00:00:16] Speaker 2: - Welcome back to theCUBE's NYSC Wired Studio here in the Buttonwood Podium, overlooking the Options Exchange. It's a little after hours, but I'm really excited to have John Rose here. He's the CTO and Chief AI Officer of Dell Technologies. Sir, good to see you. Thanks for coming in. [00:00:29] Speaker 1: - Great to be here. [00:00:29] Speaker 2: What are you doing in New York? [00:00:31] Speaker 1: - I actually was here for, what was called a Radical Innovators Collaborative, or I think, I don't, sometimes I forget the C, but it's a group of people that get together that are primarily Chief AI Officers, but also Chief HR Officers. And we spent the day and evening last night, comparing notes on AI, commiserating, helping each other. And it's very interesting when you get a group of 75 people together across industries and roles, senior levels, how common many of the challenges are, and even the opportunities of what people are going after. So, I have always believed that this is a complex journey to go on and doing it by yourself is a bad idea. And so this was a chance to go spend a bunch of quality time with some folks. And then I have a dinner tonight with, with a smaller group of, of similar folks, you know, so I guess this is my, my day and a half of deep interaction with my customers, my peers, my partners, and really a chance to kind of develop a collective conscious about this. [00:01:33] Speaker 2: The whole metal NYC. So HR officers, interesting. I mean, maybe it's an obvious question, but why the HR angle? [00:01:41] Speaker 1: Yeah, I mean, I think more and more, well, here's the thing. I'm a technologist. I've been an engineer my entire career. And yet for the last six months, the two topics I've spent most of my time on have been tokenomics and the, the impact of agents, primarily AI agents on the composition of work inside of companies. And so that second one has a significant overlap with your HR organization. You're, you're, you're really talking about trying to rethink the distribution of work in your company. And it turns out that we are all saying those words, but we haven't even agreed in many cases on what work is, what kind of work gets done? What kind of work do humans do? How do we organize that work? And so this idea that agents are going to change the distribution of work, yet we haven't even agreed on the term work is problematic. So that's, I gave a little like Ted talk today around that topic, but doing that with just a bunch of technologists is kind of talking, you know, on half of the story, bringing in the chief HR officers who really care about that and connecting them to the technology, gives you a much more complete picture of that second topic. What is this going to do to work inside of companies and at the world level? [00:02:48] Speaker 2: It's interesting for years, we've had this, you know, theme and meme around the future of work. It's been, you know, people have blogs, people have started businesses around the future of work. But to your point, we never really defined and broken down what work is. We got the job, we have a job description. So how did you break it down? [00:03:05] Speaker 1: - Yeah, we, it's funny, you know, I got very frustrated with, I don't like, I care a lot about language. Words matter, language matters. Having a clear understanding of the words you're using is important to make sure you're on the same page. That's true at a technical level, but in this topic- [00:03:22] Speaker 2: - You're such an engineer. [00:03:23] Speaker 1: - I'm definitely an engineer. But in this topic, you have this kind of hand wavy, phraseology where, you know, agents are going to do work, agents are going to do jobs, the jobs of the world are going to be disrupted. And yet you're right. We never sat down and said, do we even understand what we're talking about? Do we have a common definition of these things? And so the talk I gave was really around a bunch of work that we had done on the research side. And we had asked a question about, what is the relationship between people, agents, and work? Those three things, are they related? Is it clear? And by the way, there has to be a clear relationship because we use phrases like, agents are going to do the work of people. And if you don't even understand how these are connected together, it's very hard to kind of prove that. So what we did is we went through two exercises, which were fascinating. One of them is we took every agent we had built. And you know, we've been building autonomous agents for quite a long time. And we mapped them on a, on a X, Y axis. We basically, one axis was the degree of complexity of the work, the process. So there's some really simple things, then there's really complex things that get done by agents. And then on the other side was the degree of autonomy of the agent. So doing work in a low autonomy setting is the agent just kind of following a set of rules. You know, it's kind of rule-based engines. High autonomy is letting the agents reason. Let them think, let them be creative. And so we put those two axes together. And then we plotted every agentic use case we had built to date. And there were a lot of them. And they started to cluster. That was the interesting thing. And they clustered into four areas. In the bottom left-hand side, the clustering was really where you had low autonomy and low complexity. You have an agent doing relatively simple work in a relatively predictable way. Book my travel, summarize that document. And we started to describe those as productivity agents. They're agents that are grounded in a human being. And they fundamentally do a task for you. They're agents that do work for you. Fairly straightforward. In the lower right-hand quadrant, it got interesting because they have high autonomy, but still low complexity. So what are these doing? And so we started to call them steward or hygiene agents. And fundamentally, what we realized they were doing was do the boring work for the company. So these agents are headless. They don't have a person attached to them. They're fully autonomous, but they're doing kind of boring stuff like clean up my CRM data, make my CMDB better, do a bunch of hygiene tasks, work that nobody wants to do. But now we have these tools that will do them for us, but they're not doing them for a specific person. They're doing them for the company. So they're kind of operating independently. We jumped up to the right hand. We haven't found another cluster. And these were where we had high degree of complexity of the process, but low autonomy, which basically is code for the agents are involved in something complex, but they don't get a lot of latitude to reinvent it. They're just there to make it happen more predictably. And so we started calling them coordination agents, which are basically agents that take a task that is well defined, maybe even very complex. And their job is to just make sure it happens properly. So we've used these in factories to make sure that a product gets built correctly. We've used them and we're starting to explore in the sales organization, how do you hand off an account between a sales rep and another sales rep? It seems like, you know, it's a pretty complex process. It always breaks down, but if you give an agent the job to make sure that happens, it suddenly happens really well. And then we found a fourth cluster up in the upper right. And these were what we call expert agents, high complexity, high autonomy. This is an agent doing work for the company that is complex. Give me a quote. That's a very complex process to follow. Develop software fully autonomously. And so they're very hard to build. Anyway, once we realized that, you know, there was this pattern that no matter which agents we talked about, they kind of ended up in one of those four buckets. We started to say, well, maybe what we're seeing here are the kind of work that agents do. Agents do productivity work, they do hygiene work, they do coordination work, and they do expert work. Anyway, the next part of that, though, is we said, well, if that's the kind of work agents do, does it hold if we apply that to people? And so the experiment we ran is we took 6,400 job descriptions at the company, and we gave them to a set of agents with those definitions of work I just described. And we asked the agents to decompose every single job into how much of each of those four categories of work is done within the job. And they went on their exercise, and what they immediately disclosed to us is there's a fifth kind of work that we didn't see. It's called human element work. It's the other stuff. It's things agents don't do at all. It's interpersonal communication. It's empathy. It's personal relationship skills. All of these things are part of work. They aren't things agents do. But if you're going to look at work holistically, you can't just look at the four things agents can possibly do. You have to look at all the things that define work. So we added this human element. Went through 6,400 jobs and found some interesting things. First is, every job we looked at had at least three kinds of work involved in the job. And that led us to start to define job. Like we use this word job and work interchangeably. They're not interchangeable. A job is a container of work that a person does. And in that container, it almost always includes multiple types of work-- productivity, hygiene, coordination, expert, and human element work. If you took your job apart right now, you would realize that it's some combination of them. You're probably doing some productivity stuff to send emails and relate to people. You probably have some hygiene stuff you have to do. I actually do, yeah. Yeah, you're definitely coordinating things. You just met me here, got me up here. You have expertise in an area. And guess what? Human element, kind of a big deal in your line of work. You're actually doing all five of them in your job. Some jobs only have three, some have five, but it's repeatable. You can see this pattern. And anyway, you take those two things and munge them together and suddenly say, now I have a common definition of work and even the five work types that I can use to describe what agents do or what people do. And that leads us to the conclusion that if that's true, then agents are not in the business of taking your job, because there's no agent in the world that can do all of that work that a job is defined as. What they are in the business of doing is taking work out of your job, moving up below the machine line. And when they do that, they force your job to be redefined. If suddenly you're not doing productivity, hygiene, and coordination, and you're only left with expert and human element work, which is probably a pretty good job, you can't not change because you have all this white space that you used to do all this other stuff, but now agents are doing it for you. So the act of you in your job now biases and concentrates on the things that are left. And so anyway, this is all research. We found that it seems to map to every job that we explore. We haven't found a sixth type of work. But what it started to do is get people thinking in a lot more detail about if you're going to use the word job or work, know what you're talking about. But more importantly, the biggest conclusion was knowing what kind of work happens in your company in class, in these kind of classifications, is essential because it creates a common language between what agents are going to do and what people are going to do. And if your goal as you move into agentic is to rebalance work between those two things, if you're not even using the same language and you don't even define it the same way, you have no hope. But if you do, it becomes really clear what parts of the jobs are going to move to agents, which ones aren't. And that, for the first time, gives you clarity about what the jobs of the future look like. [00:10:27] Speaker 2: So this is fascinating. This is like a TED talk in the cube. I love it. So thank you for going through that. So obviously, the productivity piece and the hygiene piece, take that off my plate tomorrow. And the coordination even, right? I mean, agents are perfectly capable of doing that. The fourth one, the expert agents are interesting. Yeah. Because, you know, some of that I want to keep to myself. But if the agent is more expert than I am, okay, I got to let go to grow. [00:10:48] Speaker 1: That one's a fair game right now. Today, our experience is that it's really hard to build expert agents because you need deep digitization of knowledge. You need knowledge graphs that are very, very accurate. You need a lot of institutional knowledge transfer to build an expert agent. And many of them are very complex systems. The only way to really deliver an expert agent is actually not a single agent, but what's called a multi-agent system. Those are more complex. That is just simply a moment in time. We will eventually and we already today have early expert agents. And so more and more, some of the expert work will probably fall off the plate. More of that will be done by agents. The funny one is that fifth one, human element. I don't envision that going away at all. In fact, I couldn't imagine a world where I send a robot in to sell to a customer. You know, a Fortune 100. You have to have a human. [00:11:34] Speaker 2: Or I'm interviewing a robot. Yeah. [00:11:35] Speaker 1: If I send my robot in to do this interview, that would be really boring and kind of weird. And so human relationships are what humans are actually very good at. And we can see is the trend is more of the work that's kind of behind the scenes gets absorbed and done by machines. But what's left is really fulfilling and interesting. And most of it is next generations of expertise or insider judgment. And some of it, a big portion of it, probably the majority, is human interaction because we are doing this not for the purpose of enabling AIs. We are doing AI to enable humanity. [00:12:06] Speaker 2: So that tacit knowledge that you talk about in the fourth category, that sort of expert agents, understanding sort of intent, learning from exceptions, the like. I think that's going to take a while to evolve, you know, but ultimately that is the promise of AI, isn't it? Yeah. Here's my question. Does attacking was clearly attacking one, two, and three productivity, hygiene and coordination. Does that and if so, how does that change the organizational structure in that org chart or do we have to get to that fourth level to flatten the organization? [00:12:44] Speaker 1: The other big conclusion we had when we did this work, when we finally could see all the data, is we realized that what we were looking at was as you move into agentic, because every job includes those three, at least some portion of the three things that agents can do really well now. And what that means is every job changes. You not only have to change your org chart, you have to change jobs, because if your job description, yeah, everybody has job description, that's why that's what we used your job description that today said you're going to have to be able to send emails, you're going to have to write summaries, you're going to be able to present stuff, and you're going to have to be able to coordinate meetings, if those are no longer in the job description, is it even the same job? And the answer is no, it's a new and different job. Once it's a new and different job, well, the next thing that happens is how do you build the organizational chart to know how many people you need to do that job going forward? And if the job has fundamentally changed, you may need fewer or you may need more people to do it, but the organizational structure changes. And so this kind of gave us, you know, an insight that said, every job in the world is going to change as a Gentic rolls out, not because the agents will do the job, but because they will extract types of work from every job, and that will free up space for that job to be redefined. And the consequence of that is that as you deploy a Gentic, there is a massive organizational design exercise that you have to think through because if you don't do an organizational redesign, you will have a whole bunch of jobs that used to have 100% work inside of them that now have 30% of the remaining work and nothing to do with the 70% unless you redefine the job. And so it just made it very clear to us that there's a profound change coming. The other interesting thing that we concluded, though, is everything I just described kind of describes agents not as anthropomorphized beings, but as just tools, which is exactly what they are. And so one of the things I've said before is that with the exception of what we call concierge agents or super agents, which are agents whose their whole job is to be the interface into humanity. They act as kind of a buffer between all this stuff and people. If you take those off the table, every other agent makes about as much sense to put it into your org chart as it would have to put your word processor in the org chart five years ago. It's just a tool. It's not something that belongs in your org chart. In fact, its job is to take work out of the org chart and move it down into the technology layer where it's just done for you. [00:15:02] Speaker 2: But don't these tools have intelligence? And the reason I ask that is because there's a theory, organizational theory, and some of the deep thinkers are saying, we're going to change the structure of the organization, not around hierarchical org charts around people. It's going to be organized around intelligence. And I was like, okay, what does that really mean? What I'm hearing, what I'm learning here is it's really intelligence around work. [00:15:26] Speaker 1: Yeah, around work. Not around jobs. Jobs are this crazy human factor that basically you take a bunch of different work, different types of work, and you wrap them around a person and call that a job. And the reality is what's really happening in companies is work is getting done. But we need those structures because we have to organize work around humanity. When you don't have to do that anymore, when the work is not done by a human being, it becomes way more efficient. You can aggregate work. You can have a set of agents providing the coordination services for thousands of people, and there's maybe only one agent doing that. They are polymorphic. They can change their behavior. An agent that is doing coordination right now for one set of people, tomorrow can be doing the productivity work for them potentially. They can change their behaviors. And so it just gives us a whole new set of constructs to kind of efficiently do work. You know, why do we talk about productivity with agents? It's not because they're adding more arms, legs, and hands, and bodies to the organization. It's just they're simply more efficient in organizing and processing work. The minute you start calling it a job, you artificially constrain it into something that doesn't even map to what it really does. And so anyway, this is very provocative, but just full disclosure, this is research. It does seem to hunt. It's only about three months old. But so far, when we test it, it does seem to help us connect the dots that, yes, we are going to redefine work in the world, and we are going to redistribute it. But now we know exactly what kind of work. We know what categories are prone to do that. We know what ones aren't. And that informs better our decisions about what jobs actually emerge for humanity. And there are quite a lot of them. They're just all different than the jobs we have today. [00:16:58] Speaker 2: Three months old in this market means it'll be operationalized by the fall. We are already operationalizing this. Okay, so that's my next question is how do you affect a hundred plus billion dollar company? How do you affect this change? Do you start from the edge? How does that work? [00:17:14] Speaker 1: Well, in this particular category, one of the biggest benefits of having this framework, this discussion is that now we realize that every agent is not a snowflake. If some agents are optimized for hygiene and other ones are optimized for productivity and other ones do coordination, these are different tech stacks. They're actually different kinds of agents. They need different kind of data. But funny enough, if I solve for one of them, if I can build a hygiene agent, and we've done this, I build hygiene agents that do like CRM cleanup, and then I use exactly the same architecture with different data to go clean up my CMDB or to go do another task. And so I create this, I go from being every agent is a new adventure and a bespoke snowflake to really I'm building a platform that supports three, possibly four kinds of work. Make that industrialized, make it scalable, make it repeatable, and suddenly almost every problem I find that I want to move into the agentic world, I'm not having to invent something. I'm just using one of these characteristics or templates to basically solve that problem repeatedly. For me at our scale, I can't build a snowflake for every agent. That would be nuts. But if I solve for hygiene agents, I probably have a thousand hygiene problems I can go solve, and now I can do it in a very repeatable way. Same thing with coordination, same thing with productivity. And the reality is it just makes it more architecturally doable once you have this kind of normalization, because you're no longer thinking about every problem you're solving is a brand new adventure that you have to figure out. That just is a path to move slow. But once you create the patterns, you can move fast, because you're really doing almost a no-code configuration to deploy your next hygiene agent. The first one's hard. And then once you have the platforms in place, the next thousand are pretty much just rinse and repeat. [00:18:52] Speaker 2: And an accelerated cycle as a flyer. AI is speed. If you're fast, you win. This has been, absolutely, this has been fascinating. I want to ask you about AI factories and what your customers are doing, because all of this requires tokens. And you can either buy them from a token generator, or you can make them yourself. [00:19:12] Speaker 1: Well, let me give you an example. I'll stay with this thread and talk about AI factories. Now, you know, we are very much believers in hybrid AI. We believe that there should be choice. You should have options of where your tokens are produced and where they run and where your data lives. The idea that there's only one place to do this is nuts to us. This agentic discussion validated even further. And the reason it did is that when you think about those types of work, hygiene work versus productivity versus coordination versus expert, what is the economic value of them? Are they all exactly the same? Like, how much value is in, you know, I don't know, summarizing a document? How much value is in cleaning up one CRM record? And the reality is those are relatively low-value tasks. But how much value is in coordinating and optimizing the production of a million-dollar product? Now, I'll spend money on that. That's worth it. Or building a fully agentic developer or a CNC operator or some very advanced expert agent. And so suddenly you start realizing that these agents, while there's a pattern, they are not equal in terms of their economic value they create. Now, why does that matter to an AI factory? Well, the AI factory is about choice. I want to have different places to run them. If I'm building a hygiene agent, one of the tricks we do is, you know, I might run my hygiene agents in my AI factory in the white space of the GPUs I'm not using in nights and weekends because they're not time sensitive. And the reality is then I suddenly get capacity for free because if I used tokens on an API to do the same thing instead of costing zero, it might cost me $12 to solve a 50-cent problem. Because it breaks the consumption. It breaks the consumption model. But on the other hand, if I have an expert agent that's got, you know, millions of dollars of impact, I want to have the option of using the most advanced frontier models and be able to use them as aggressively as possible. The beauty of the AI factory is it's all of the above. You don't have a single choice. You can have open models running on-prem. You can have frontier models running on-prem like our relationship with Google. You can have virtual private clouds that are able to securely access the data that you control and basically use a secure cloud environment. You can have APIs running through secure environments and pass data back and forth between them. All of the above is perfectly fine. But if you didn't have that choice, if you only had one of those choices and you tried to run every agent using a single approach to token generation, you would maybe get lucky once and get the economics right. And every other time it would be a subpar exercise because you'd either be too expensive, too slow, too insecure, some dimension would happen. And so having that ability to build a hybrid AI environment and have choice about mapping not just your AI but the agents doing work to the place that is best suited for them from a security and economics and performance level is essential. Because agents are not homogenous, even if you only believe there's four categories, those four categories all have different requirements from infrastructure and having a single approach through a third party that you have no control over will never get you there. [00:22:06] Speaker 2: Are CIOs rethinking, or CFOs even, rethinking CapEx, broadly defined? I mean, you can turn CapEx into Opsys, but rethinking CapEx because it is how you get intelligence and how you generate productivity? [00:22:19] Speaker 1: Yeah, I mean, six months ago people were doing it, but maybe less vigorously because the only reason they were doing it was primarily they believed there was a control and risk problem, which there still is. Having secure private infrastructure as CapEx gives you a very different security model than going to a third party, what happened in the last six months is tokenomics, people suddenly realized that the API based token production services are now usage based, they're not flat rate. And so suddenly, as you use more advanced models to do more advanced things, if the only way you can do that is over an API into some service that's out of your control, the economics get quite complex. Now, the reality is there are perfectly good things that you should use that for, and there are many, many things and many, many types of agents that should never go near there and would be better served by an open model running on-prem or an on-device model or now a what's called a personal agent framework like NemoClaw, OpenClaw, Hermes, those will eventually become more secure and stabilized. The reality is it's choice, and it's choice now not because of just where you run it from a security perspective, which has always been true, but it was never quite enough to get everybody excited about the fact that hybrid matters. Economics gets people to the table. CFOs now care about hybrid AI because they know that it's a way for them to control the economic explosion that could happen if they don't have an ability to selectively place where they consume their tokens and have both CapEx and OpEx models. [00:23:44] Speaker 2: And it gives them the flexibility, as the example you said, you could just run the job on the weekend at a spare capacity. John, I know you've got to go. You've got some guests coming, so I appreciate you coming in. Dave, good to see you. Great to see you. Thank you. And thank you for watching. This is Dave Vellante for the Cube NYSE Wired, and we'll see you next time.