I Went to the Biggest AI Infrastructure Conference

[00:00:00] Speaker 1: AI agents are everywhere, but almost no one's shipping them reliably, and the reason for that is pretty simple. Most AI agents now are just Python scripts running in a loop. They work great in a demo, but as soon as it gets to production, all kinds of things fail. An API times out, a server restarts, your process dies at step 6 of 10, you name it, all kinds of things happen, and then you're stuck writing all type of retry logic, state management, and those are things that you're definitely going to get wrong. Now, I've dealt with this firsthand. The AI part usually works fine, but it's the orchestration around it that can become an absolute nightmare. Think about long-running state, failures, rate limiting, authorization issues. All of these come up in production, and you need a way to handle them. Now, this is what durable execution solves. Instead of you handling all of this state and writing all of the retry logic, a platform can handle it for you, and then you just worry about building the AI. If something crashes, it can pick up exactly where it left off with no retries and no duplicated work. Now, Temporal is the platform behind durable execution. It's open source, it has SDKs in over five different programming languages, and it's free to use from your own machine. Now, this handles retries, rate limiting, pretty much anything as it relates to orchestration of AI agents, and it makes it really easy to build and deploy agents at scale. In fact, it's already used by companies like OpenAI, Netflix, because of the capabilities it has. Now, today I'm at Temporal Replay in San Francisco. I'm going to dive into this more in-depth. I have a ton of sessions to attend. Let's get started. So, I'm just on my way to my first talk here on AI agents in Python using Temporal. It looks pretty exciting. Let's check it out. Now, today I'm at the Temporal Replay Conference here in San Francisco. I've got all kinds of exciting talks about AI agents. Let me bring you along and show you what I learned. All right. So, day one was packed with a ton of workshops related to Temporal and their SDKs. Now, naturally, I attended the Temporal for Python session where we learned about how to write durable AI applications. Now, we ran through five to six different demos where we went from writing AI agents from scratch to using all of the Temporal primitives to make our applications traceable, observable, and most importantly, durable. And by the way, what does durable even mean? Well, it means your workflow state is never lost. Now, normally, if your app crashes, a server restarts, or some API call fails halfway through, you need to manually handle the retries, the state recovery, and all of that complexity yourself. Now, Temporal solves this by storing the entire workflow state inside of the Temporal server. So, your code can pause, fail, retry, or even continue running days later exactly where you left off. Now, your workers execute the code while the Temporal server acts as the source of truth that keeps track of every step in the workflow. Now, we'll dive into that more later because after my session, I actually found another tech YouTuber, Ona Codes, and here's what he had to say. All right, guys. So, I'm here at Temporal Replay in San Francisco. Look who I met. We got Uma Codes here. And, uh, anyway, say hi, man. Hello. How's everyone? Hope everyone's doing good. [00:03:32] Uma Codes: Yeah. How are you enjoying the event so far? I am loving it so far. I've been to a few [00:03:37] Speaker 1: workshops and I am learning a ton. Yeah. Yes. What would you say the number one thing is you've learned so far? [00:03:43] Uma Codes: Um, I think how much they abstract from us so we don't have to worry about it when we design our systems. So I'll say that's the most interesting thing that I've learned so far. Yeah. And, uh, do you have a lot [00:03:53] Speaker 1: of experience building AI agent stuff? Do you do it on YouTube or, or is this kind of the first one? [00:03:58] Uma Codes: I have experience building AI agents, but in like very confined workflow. Um, not in the context of [00:04:06] Speaker 1: Temporal. Gotcha. Gotcha. Okay. So this is your first time using Temporal. Yeah. This, this would be my [00:04:11] Uma Codes: first time. I have heard about them a lot. Um, like I said, the whole claim is what if your code or your process doesn't fail or never fail. So it's super interesting to actually see how that process [00:04:20] Speaker 1: works behind the scenes. Yeah. A hundred percent. Yeah. Thanks man. And the day continued with plenty of other workshops covering some of the other SDKs that Temporal has available. And they even dove into their Nexus framework for more advanced and distributed workflows, but rather than just show you B roll here, I want to give you a quick demo so you can see how this actually works. All right. So I'm on the computer here and I'm going to give you a quick demo of how Temporal works. I'm just going to show you a finished workflow. We'll briefly go through the code, but of course, there's a lot more to dive into here to really fully understand it. So the basic idea of Temporal is that you have kind of three main components. You have a dev server, which I'm running right here, which is kind of handling all of the message brokering between like the actual AI running and the different clients. We have the worker. This is where you'll actually run something like an AI agent. And then you have the client. This is where you send some kind of requests, right? To initiate some type of workflow. So if we just do something basic, right? Like I submit a request saying, what is the weather in Dubai? We'll see that if we go over here to the Temporal dev server, which is running right now, we can see we have a running task. We can click into it and we're able to view a timeline of all of the different events that are going on. Now, in this case, it ran quite quickly, but if it was being slow, we could see what activities were slowing us down. We can see exactly how long everything took, and then we can step through this and see all of the results from any given step. And this is what I was talking about with the traceability and observability. This really allows you to have better insights into what your AI agent is doing. So you can see, we called GPT four. Oh, this was the systems instruction. We had, we had some input, which is what was the weather in Dubai. We can see all of the arguments that went through here. And then the benefit of this is that if this were to fail or there was some bug or something, we wouldn't restart the entire workflow. We would just sit here and wait until this was kind of rebuilt or ready to run again. And it would continue from the other task. Now I can show you this with a failure, but I don't want it to take too long. And then if we go back here, let's just say I run, you know, another task right here, give this a second. Let's refresh. You can see a new one comes up and is running and we get a full log of everything that's going on. Now, all you really have to do in order to use temporal here is just write a little bit of code. So for example, if I go here, this is the start workflow file. You can see this is kind of, you know, representing the client. If we go to the tools workflow, you can see this is a particular workflow where we kind of have an AI agent that has access to a few different tools. And then if we have a look at these tools, we run them as something called an activity. And whenever we put a tool in an activity, it automatically makes it durable, which again, gives it all of those benefits that I talked about before. Now, again, I don't want to dive into too much of the code here, but I just wanted to show you a quick demo of how this actually works. The main thing here is this temporal dev server, where you can have all these activities, schedules, batch workers, nexus, all of this kind of stuff. You can see the workers that I had, you could have multiple of them. You could have namespaces. And then of course the workflow itself, where you're executing kind of a run of an LLM or an AI agent. Anyways, let's get back to the video and you'll learn a little bit more about it. All right, guys, we're just wrapping up day one at the conference, just did some workshops today, learned about durable AI agent execution, super cool stuff. I would have thrown some demos in earlier. So hopefully you guys got some information tomorrow. We've got a bunch of keynotes speakers, other creators. I'm going to talk to stay tuned. See you there. Now we move on to day two of this three-day conference where we started the morning nice and early with the official keynote. Now there was a lot discussed and announced here. So I want to go over what I found most notable. Now, number one, Temportal is used by almost all major AI companies and it's essential to their infrastructure. OpenAI, Mistral, Emergent, Docker, Replit, Retool, you name it. They are probably using Temportal and they're using it more than ever before. Now, number two, Temportal has seen massive adoption as become the de facto infrastructure for managing AI applications at scale. Now OpenAI themselves grew their usage by over 60 X in just the last year. And then number three, they announced three new capabilities. We had serverless workers, standalone activities and workflow streams. I'm not going to go into details on all of this as only makes sense if you're a preexisting user, but I can say that the crowd was as energetic as I've ever seen for a tech conference full of a bunch of developers when these were announced. And for those of you that are still a little bit confused on what Temportal is, I actually met up with another tech influencer, Chris, who gave her best shot at explaining it. Let's hear her. Chris, nice to meet you here at Temportal. We're going to have a quick conversation. Can you give a [00:09:19] Speaker 3: quick intro to the audience? Hi, everyone. My name is Chris. I'm a tech content creator as well. And we met at this conference. It's really nice meeting you by the way. My question is, a lot of people don't know [00:09:29] Speaker 1: what Temportal is that's watching this video. Could you give us a quick high level explanation of your [00:09:34] Speaker 3: understanding of what Temportal is? My quick understanding, like simple, in simple terms, I would say if you are building anything, like if you're a vibe coder or you're a software engineer, you know that almost any application breaks and Temportal can help you prevent that and catch errors early and handle certain things on your behalf. So it's a very useful tool to have if you're building anything [00:09:57] Speaker 1: pretty much. For sure. For the technical people, it's like durable execution for, you know, AI agents, but observability, logging, retries, long running tasks, delegating to an IO, you know, something like that with AI agents. So that she put it a lot simpler than I think I described it. So what do you think so [00:10:14] Uma Codes: far? Day two? Day two. A lot of hype. Yeah. That has been backed up by evidence actually. Open AI with your 600 million, along with a bunch of other companies. So, so far so good. Yeah. Let's keep [00:10:30] Speaker 1: it going. Now, what he was just talking about there was open AI and Temportal's partnership and what we heard from the manager of the applied infrastructure team at open AI when he was discussing during the keynote. Now I'll leave a link to the recording of the talk down below as we roll into day three, day three, again, started bright and early with another keynote, but this time focused more on a technical deep dive of Temportal and some of their newer features. Now, unlike many other conferences, they actually showed a real functioning demo on screen. They stepped through the code and they showed us logs, dashboards, and everything else that most other companies just try to hide for their dear life. Now I don't have enough time to really dive into all of the details here, but it was a nice change of pace from the pure marketing hype that you usually see at all of these conferences. Now you may notice the change of background here, and that's because I had to leave pretty early on in the third day to take my 16 hour flight. Yes, 16 hour flight back to Dubai. So I'm here now, and I've had some time to kind of reflect on what I've learned. And I want to leave you with this. Now AI is not going anywhere. It's being integrated deeper and deeper into our daily lives. And as a developer, we're being asked more and more to build it into applications. Now, the issue is that AI breaks, right? It fails, it times out. And the more we try to scale it, the more of these issues we run into. Now, this turns our apps into maybe 5% code for calling LLM APIs and 95% for managing all of the issues that can occur. And this is the exact issue that temporal solves. And it's why so many companies are scaling their adoption of it right now. And this makes sense, given that it's free, open source and available across all of the major languages. So you can adopt it with no risk whatsoever. You don't need to pay for anything. If you haven't already, give it a try. And I'm sure once you do, you're going to see what I'm talking about here. And you'll understand why all of the top companies are racing to adopt it. Anyways, guys, that's all I have for you in this video. I hope you enjoyed this. And if you like this style of video, definitely let me know. I'm happy to make more. I'll also be doing a temporal deep dive soon. So stay tuned for that, where we actually dive into the code and I show you exactly how it works and how you can build with it.