Today’s episode features an interview between Matt Trifiro and Simon Crosby, CTO of SWIM.AI. In this interview, Simon discusses how SWIM is solving the problem of stateless computing with its edge intelligence software that focuses on edge-based learning for fast-data and continuous intelligence.
Today’s episode features an interview between Matt Trifiro and Simon Crosby, CTO of SWIM.AI.
Simon’s impressive resume spans two decades in technology and includes highlights such as co-founder and CTO at XenSource, co-founder and CTO at Bromium, and CTO of Data Center and Cloud at Citrix.
In this interview, Simon discusses how SWIM is solving the problem of stateless computing with its edge intelligence software that focuses on edge-based learning for fast-data and continuous intelligence.
Key Quotes
“We’re moving from a world of big data, where you could reasonably store it all and then think about later, to one where data flows are boundless and you need to process it on the fly. You need to continuously process and analyze streaming data to get continuous intelligence to make your organization more responsible. But the big change is that, whereas before you could reasonably think about big data as a way to store stuff and then analyze it, now you can't.”
“The cloud has been tremendously successful really because of two big things. One is rest, which is stateless computing, and the other one is databases…But the problem with that is that stateless computing is a million times slower than the CPU. A million times slower. That means the difference between hours versus milliseconds in terms of getting results."
“In this era, we have data flows which are boundless. They never stop. And the data within them is of ephemeral value. So you can't store it and get to it later because later it’s useless. What you care about is that some representation can predict what's going to happen next, but you don't care about the past. So we have ephemeral data value and infinite data, and you need to compute on it continuously…and that's a whole new approach, algorithmically and mathematically.”
“We shouldn't apply AI to situations where the marginal cost of being wrong is high…if the cost of being wrong is very high, like somebody dies, just don't do it. But if the cost of being wrong is marginal who cares? Like my Uber may stop at a red light, who cares? Right? We're still better off…There are tons of opportunities for making the world better, where if you're wrong, you don't make it worse.”
Sponsors
Over the Edge is brought to you by the generous sponsorship of Catchpoint, NetFoundry, Ori Industries, Packet, Seagate, Vapor IO, and Zenlayer.
The featured sponsor of this episode of Over the Edge is Ori Industries. Ori Industries is building the world’s largest edge cloud. Their products power the next generation of intelligent applications through unparalleled access to major communication networks worldwide. Ori is laying the foundations for application developers to seamlessly deploy to uncharted edge computing infrastructure across the globe. Learn more at ori.co
Links
[00:00:00] Matt: [00:00:00] hi, this is Matt Trifiro CMO of edge infrastructure company, vapor IO, and co chair of the Linux foundation state of the edge project.
[00:00:07] Today, I'm here with Simon Crosby cto @swim.ai. This is the second time we've had a conversation like this. The first time we spoke, the audio file was corrupted, but the conversation was so good. I asked Simon if you'd do it again. And he so kindly agreed. One of the things I like about him is that he is delightfully opinionated and he backs up those opinions with deep domain knowledge.
[00:00:25] We're going to talk about Simon's career in technology, the edge AI he's working on it, swim. And of course his views on the future of edge computing. Hey Simon, how are you doing today?
[00:00:35] Simon: [00:00:35] It's great to be back. Thank you.
[00:00:37] Matt: [00:00:37] Yeah, so for the listeners who didn't get to hear the first recording, how did you even get started in technology?
[00:00:44] Simon: [00:00:44] Oh, I have a PhD in computer science. I used to teach at the university of Cambridge and then I started doing startups. So I'm on number five.
[00:00:57] Matt: [00:00:57] So Cambridge is in the UK, but your accent doesn't [00:01:00] sound British.
[00:01:01] Simon: [00:01:01] that's right. I grew up in South Africa,
[00:01:04] Matt: [00:01:04] Yeah. And, w what, what was your field of study in your, in Cambridge?
[00:01:08]Simon: [00:01:08] computer science and mathematics. So I guess in another era you could call me a data scientist.
[00:01:16] Matt: [00:01:16] And, although your career, you're now back doing data, but, you did some virtualization technology. So, so what drove you, to become an entrepreneur?
[00:01:25] Simon: [00:01:25] You know, the academic world, computer science is kind of cool. You get to do fun stuff, but it's a little bit rarefied. And if you really believe passionately in a thing, some tech, you know, you might as well get out there and try and change the world. Well than just writing papers about it and publishing.
[00:01:45] And so I helped out and I did it
[00:01:48] Matt: [00:01:48] That's great. and what, so, so what are you passionate about today?
[00:01:53] Simon: [00:01:53] Oh, right now I'm passionate about, you know, a big change that I see coming, which is relevant [00:02:00] to your world and mine, which is that we're moving from a world of big data. where you could reasonably store it all. And then think about later too, one way data flows are boundless. And you need to process it on the fly.
[00:02:16] So you need to continuously process and analyze streaming data to get continuous intelligence, to make your organization more responsible, whatever it happens to be. But the big change is that. Whereas you could reasonably think about big data as a way to store stuff and then analyze it. Now you can't.
[00:02:38] Matt: [00:02:38] Okay. And why can't you
[00:02:41] Simon: [00:02:41] There's so much of it. Okay. I'll give you an example. One of our customers is a mobile provider and we are currently dealing with about 20 petabytes of information per day. Okay. That's it'll probably get to about a petabyte per hour. That's a lot.
[00:03:00] [00:03:00] Matt: [00:03:00] A petabyte an hour. Okay. So let's scale a petabyte for the listeners. How big is a petabyte?
[00:03:05] Simon: [00:03:05] Okay. It's 10 to the 15 bites.
[00:03:09] Matt: [00:03:09] That's a lot of zeros every hour, and this is just one mobile operator. Probably one country.
[00:03:18] Simon: [00:03:18] Right. and the application there is to continually optimize the connectivity between hundreds of millions of mobile devices and the radio network. So what you want to know is. Where every device is where it's going. It needs to predict where it's going. You want to know how to tune the radios and how best to allocate the capacity amongst all the devices that connect, access the network.
[00:03:47] Matt: [00:03:47] And, perhaps relevantly the, the status of a modern cellular network. I mean, changes every millisecond or even faster.
[00:03:57] Simon: [00:03:57] That's right. And there are a few [00:04:00] problems that remain to be solved. For example, in 5g, there's this great thing called slicing, which allows them to sell private networks to enterprise at the edge, but, you know, assignment of capacity to slices that dynamically, isn't an unsolved problem.
[00:04:19] Matt: [00:04:19] so let's so how does, swim AI, address this problem? what does it do?
[00:04:25] Simon: [00:04:25] So the cloud has been tremendously successful because of really two big things. One is rest, which is stateless computing, any old civic will do. And the other one is databases. So if you think about AWS Lambda, for example, You know, and the old server can take the request, load my code, and then my code will update the database.
[00:04:51] Yay. But the problem with that is that stateless computing is a million times slower than the [00:05:00] CPU, a million times slower. Okay. And so that means the difference between hours versus milliseconds in terms of getting results. And so if you adopt an architecture, which is stateful. When dad arrives, you transform it.
[00:05:19] You transform the as applies passes were in memory at CPU memory speed, which is a million times faster than this risk based approach. So staple computing is a fundamental approach to what we do. and then beyond that, The transfer. What you have to do is change the stack so that, you deal with this notion of boundless data.
[00:05:46] So we're in this era, we have data flows, which are boundless. They never stopped. And the data within them is a femoral value. So you can't store it and get to [00:06:00] it later because. Letter to users. You really don't care. The light was green last Tuesday morning, right? At eight, seven and 13 seconds. You just don't care.
[00:06:10] What you care about is that some representation of the intersection can predict what's going to happen next, but you don't care about the past. So we have ephemeral data value and infinite data, and you need to compute on a continuously. And so the notion of learning and analysis, learning competency and prediction is all based on changes to the idea that you have complete data.
[00:06:41] You have to continuously compute on infinite data and that's fun. It's a whole new approach, algorithmically and mathematically.
[00:06:51] Matt: [00:06:51] So, you said computer. On this data, what is, what does that look like? I mean, what is doing the compute and what's it computing, and if I'm [00:07:00] tossing the data out because it's ephemeral, but I want to improve my ability to predict. where does that improve prediction go?
[00:07:08] Simon: [00:07:08] Yeah. So, so that's your spot, all the key things. what swim does is effectively. Build a model on the fly from data. So for everything that's represented in streaming data, we create what you might think of as digital twin, but it's way better. We call them web agents, which represent that thing. And these are stateful objects that concurrent.
[00:07:36] So they continuously compute on their own data. They're in memory somewhere in a cluster. Or maybe a distributed application and they can compute continuously. So they actively go out and acquire their own raw data and then incorporate that into what they know. Okay. So they analyze it, learn and co and predict for [00:08:00] themselves.
[00:08:00] Right. So. For everything that's represented raw data. We end up with a concurrent object called web agent, which continuously processes its own raw data and analyzes, learns a predict from that. And then it does this other cool thing, which is linked to other things that is related to certain the case of traffic, which I've used in the example of already, an intersection would link to all of its sensors.
[00:08:30] Okay, that's pretty all this containment, but it also could link to other, in sections, in a specific city and linking gives this concurrent object, the ability to see the state of other things. And so we're building a graph like a graph database, but building it in memory where these verses in the graph can continuously compute based on what they can see.
[00:08:58] And they compute [00:09:00] and then stream the results.
[00:09:02] Matt: [00:09:02] So, I'm imagining. This, you know, all these little web agents and their connections between each other. And I, you know, I'm not an AI expert, so this may seem overly fantastic, but it seems an awful lot like how the brain works. I mean, is it modeled after the neurons in the brain? And is that something, or is that just a coincidence?
[00:09:22] Simon: [00:09:22] It isn't Kenny, you know, comparison to it as you pointed out. but so there's a nice way to think about like this, which is that if I said to you, Hey Matt, do you like blueberry muffins? You know, you know the answer to that because you've learned to your life, you didn't have to fund your mom.
[00:09:42] Okay. Which is the equivalent breaching out the database. And you didn't have to go through all the blueberry muffins you've eaten in your life, you know it okay. And so these web agents are concurrent entities, which learn from their own data. Right. And so what you find is [00:10:00] these amazing emergent behaviors, from collections of web agents, and these are self assembling complex applications.
[00:10:09] The cool thing about it is this that let's go back to traffic again. As a programmer, all have to define the relationship, say between an intersection and its sensors, and it's the Cindy for one in section. And then I can scale this thing. Data will scale the application at runtime using the entities in the dates to create new web agents.
[00:10:35] And, so the actual application will just build itself and run. Okay. And that's okay for a small city, like Palo Alto where I have a few hundred intersections, but Houston has tens of thousands. Okay. Or in case my mobile carrier, I have, you know, hundreds of millions of devices.
[00:10:57] Matt: [00:10:57] Yeah. So let's, let's put a, [00:11:00] let's put a pin in the, I want to come back to emergent behavior, but let's talk about the traffic light example. Cause I think it's kind of interesting. So you've got this installation in Palo Alto. and, as I recall you were there's, a lot of data that you potentially could pull off a traffic light, actually.
[00:11:15] Why don't you explain how you retrofit. city and how the date of reduction happens. Cause it's really interesting.
[00:11:22] Simon: [00:11:22] so let's talk about edge in general, right? Because you know, we dealt with old infrastructure, like traffic stuff. Now, these. The traffic infrastructure is 30 years old. And so the debt is horrible. It's, you know, the luminous.
[00:11:37]Matt: [00:11:37] there is no ethernet port on the back of the light.
[00:11:39] Simon: [00:11:39] Okay. Right. and so we'll be getting is voltage transitions between relays on some legacy thing right. In the city data center. Right. And so. we tend to get access there because they have sent those interfaces. but there's tons of it. You [00:12:00] know, a city like Las Vegas is 60 terabytes a day. Houston is hundreds of terabytes a day.
[00:12:07] Okay. And so the first thing is that. This Gosling form of information, which is volunteer transitions. And so on gets transformed into this light is green. Okay. And the fact that this light is still green, you can just throw away, right? And so you get massive bed reductions by moving the competition associated with voluminous stuff, close to the source of data.
[00:12:37] All right. So there's huge opportunity for edge computing there, which is to move the computation associated with filtering better close to the sources.
[00:12:48] Matt: [00:12:48] where does the transformation from, voltage relay to green or red or yellow happen? Does that happen on the web agents?
[00:12:57] Simon: [00:12:57] yes. So there'll be web pages, [00:13:00] which for every. thing, every sensor in the city. And, but those will naturally migrate when the cool thing is swimmed, as it moves, web agents close to wit to the right place to compute. Okay. And that right. I said defined by their desire for memory CPU, additional resources like GPU's and their proximity to data source.
[00:13:28] Matt: [00:13:28] So, let me make sure I understand that. So, part of the swim dot a secret sauce is you are orchestrating all these web agents across potentially multiple machines and multiple locations. And you're using some criteria, like you said, access to availability, to CPU available, the GPU, things like that, to do that kind of orchestration.
[00:13:51] Simon: [00:13:51] Yeah. So scrim moves them around in real time. So L magically move around within a cluster. And so [00:14:00] for example, traits associated with web agent, like learn, predict might migrate into the cloud. So in the case of traffic, the. Dealing with voluminous stuff at the edge will happen at the edge close to the source of data, but learn and predict might happen in a GPO attached instance in Azure.
[00:14:25] Matt: [00:14:25] That's really interesting. So, how does. So you mentioned, okay, you've got all of these, devices out in the field. let's make it simple as say
[00:14:35] Simon: [00:14:35] By the way.
[00:14:36]Matt: [00:14:36] no. And realize that, Oh, is that right? So there are the face of the field, but all the traffic lights in Houston or Las Vegas or Palo Alto and that data, hopefully is already being aggregated by the city, in a centralized location.
[00:14:47] Otherwise I'm just gonna figure how to do that. And then you have, one web agent for every traffic light.
[00:14:55] Simon: [00:14:55] Well, the cool thing about, well, let's talk about the term web page, right? So [00:15:00] web means the interesting part of the name is web, which means that the object ID is a URI. So as they move around, we always know where to find them. Okay. So they can move around in a cluster and we still don't.
[00:15:15] Matt: [00:15:15] keep track of them.
[00:15:16] Simon: [00:15:16] Bingo.
[00:15:17] Matt: [00:15:17] Wow. Okay.
[00:15:18] Simon: [00:15:18] Okay. And so things can relocate and we always know where to find them.
[00:15:23]more than that, we always know where their API is. So if you want to get hold of a streaming app yeah. I, for a particular intersection or sensor, you know, it's URI and you just go for it. Right. And so our UIs do this too. They just go for the particular. Thing you want to look at and they get the access to the state.
[00:15:47] Okay. So, other than that web agents are actors that is the received data and they compute in it and they'll stay staple. And when you link to them, you get to see their current [00:16:00] state. Okay. And yes, to answer your question, but we're begging for insection. Has links to all the way versions for its sensors, maybe 80 or a hundred per intersection, in a typical city.
[00:16:15] And then it will link to its surrounding, intersections could have sea of lap long and a geo-fencing. In other words, every single object will link to its neighbors and they'll link to their neighbors. So the running application is a graph, fairly complex graph of, linked things have agents, each of which is linked to its own, neighbors.
[00:16:42] Matt: [00:16:42] Yeah. So all these web agents could be running on one or multiple computers in multiple occasions, multiple racks, all of that, but it is the linkages between them, is representing the physical world.
[00:16:54] Simon: [00:16:54] Yes. And you've hit them. The absolutely key thing is it at my [00:17:00] level, you don't get to say where at your level, you absolutely care about where that is. We want to exploit your proximity to the thing. To compute close to it whenever we can. But it's the application level. The person who develops the application called learn, predict for a city, doesn't need to know where that gets sold by swim at runtime.
[00:17:26] Matt: [00:17:26] Yeah. and then you mentioned, you know, earlier how the, how, what a dream, this is for the developer. and so, so what, how do I make this system do my bidding?
[00:17:39] Simon: [00:17:39] It's pretty simple. And by the way, most of the apps are open source. So, you know, you can go to Sumo, esto org and just start to play. there are ton of applications in there. you're writing a simple. Object oriented Java application. So in the case of trapping again, it would be the [00:18:00] relationship between end section and it sensors and possibly its neighbors.
[00:18:04] So that program is probably, I don't know, a thousand or 2000 lines of code and you're done, but the running application could be terabytes worth of running stuff. Right. In the case of my mobile customer. I mean, literally you were running in terabytes of memory in a cluster, which is tens of nodes, and they're distributed across the country.
[00:18:33] But that application at runtime is built from the data.
[00:18:39] Matt: [00:18:39] That's really interesting. And how is this, an, a new way of doing artificial intelligence? And the reason I ask that is, and again, I'm not an expert in AI, but any search the imagination, but I've done enough reading on convolutional neural networks and deep neural networks and so on.
[00:18:53]and this sounds. Like it has some familiar parts, but I don't feel it's the same thing. Can you [00:19:00] relate that the type of AI that you're doing with swim.ai to, you know, the kinds of AI that we typically read about and hear about.
[00:19:09] Simon: [00:19:09] Yeah. So, first of all, I'm convinced that the state of the art will always be available in open source. And so it's absolutely wrong to claim that we are innovating in the AI algorithms, right? Because white flight that, I mean, there are brilliant things out there. The right thing to do is to take a different approach to how you process the data through those algorithms.
[00:19:35] Right. And so. Typically, if you look at something like spark, which is just down the road from you, what you're going to do is take that off a disc and throw it into some predefined neural net. And problem there is that somebody has to train that thing, right. And then you're off hunting dead to push through it.
[00:19:57] The problem with that is, that it [00:20:00] takes. A, an externality, somebody to take the train and model our approach has been in all the uses we made thus far, is to use red theory, straightforward, simple unsupervised learning techniques. And by the way, when I say simple from them or astonishingly simple, like just learn your aggression.
[00:20:21]so a thing learns, it predicts for itself. The
[00:20:25] Matt: [00:20:25] So, so my web agent for the traffic light at the intersection of X and Y is constantly refining its ability to predict when it's light is going to go from green to yellow. Is that
[00:20:37] Simon: [00:20:37] And more than that, based on his own sensors and its right, which is a very simple, but quite rational view. Now, what can you do with that? In the case of traffic, what we do is we stream those predictions to customers. So they would go straight into Uber's comforter. So if a boat wants [00:21:00] to route a vehicle through the city, they get the current prediction for all future predictions for every light in the city.
[00:21:08] Okay. And they go to list to minimize transit time or something. and so it's a slightly different way of looking at the result. Instead of having one prediction for the future state of the city. I have every bit of the infrastructure to see predicting its own future state. That's kind of cool. Right.
[00:21:29] But in the case of my mobile provider, you know, I've been of hundreds of millions of. Mobile devices predicting where they'll be in the future in the near future. And then the algorithm, which is the optimal capacity assignment takes that and allocates, connection capacity across all of those different devices.
[00:21:53] Matt: [00:21:53] Wow. And, in the case of the traffic lights, again, I imagine that, you know, [00:22:00] in some future state I could also connect to the under street sensors. Right. And now my traffic light can relate to the nearby under street sensors
[00:22:10] Simon: [00:22:10] Oh,
[00:22:11] Matt: [00:22:11] its own improve its predictive ability.
[00:22:14] Simon: [00:22:14] no, we do that already. Yeah. So every time the car crosses and in street loop, we see that. And the interesting thing about this setup is, The DNNs are very simple. Maybe I, you know, maybe it's limited in terms of application, but you know what we've done thus far, relatively straightforward, single, hidden layer.
[00:22:37] When we initialize it, then it's initialized to run numbers and basically you form an input back to from a venue. Can see, imagine you're in insection you can see your own senses and those around you. So would that through the DNN, predict the future and then watch what happens. And then the inference gets [00:23:00] back propagated.
[00:23:01] Right? And you just go around that loop a bunch like. Yeah a week and you're pretty good. The cool thing about this is you can track whether you're predictions are converging or diverting. If they're diverging, then we're in deep trouble, but if they're converging, then you get to choose your operating point, which is the error that you're happy with.
[00:23:24] Or you can just leave them continue
[00:23:26]Matt: [00:23:26] is it usually pretty obvious? Which of those it's doing? Yeah. So let's talk a little bit about.
[00:23:32] Simon: [00:23:32] you'd be embarrassingly. Well, maybe it is scientists will be embarrassed by the fact that very simple methods can make the world a heck of a lot better. That is, you know, we have these very sophisticated, AI tools today for big data. Yeah, they're awesome. But actually a very modest impression on the part of lots of little things makes the world a heck of a lot better.
[00:23:59]Matt: [00:23:59] you know, that's a [00:24:00] good, that's a good life lesson. I mean, sometimes, you know, rather than installing like some clever app on your Mac, your PC, you should just buy a bunch of three by five cards because sometimes a pen and a pencil or paper are better. that's really interesting.
[00:24:14] That's a really,
[00:24:16]Simon: [00:24:16] there's one other kind of observation which goes with this, which is that, We shouldn't apply AI to two situations with the marginal cost of being wrong is high. Let me try and say that
[00:24:32] Matt: [00:24:32] Yeah, let's go. Let's go into that.
[00:24:34]Simon: [00:24:34] if the cost of being wrong is very high, like somebody dies. Yeah. Just don't do it. But if the cost of being wrong is module who cares?
[00:24:43] Like my Uber may stop at a red light. Yeah, who cares? Right? It's better. We're still better. And so there are tons of opportunities for making the world better. Where if you're wrong, you don't make it worse.
[00:25:00] [00:25:00] Matt: [00:25:00] Right, right. That's true. That's true. and I imagine, because you're committed to this company that in general, you find you're able to make it better even with relatively simple. Yeah. that's really interesting. Yeah. You know, And maybe after this, we should talk about it. But one of the things that we're doing in our network and data centers is, we are, extracting, all of the operational.
[00:25:26]information, all the OT information that nobody ever does anything with really. So we've got stream of data coming off the HVAC equipment. We've got like a hundred sensors. You know, we get the air pressure, temperature, humidity, vibration, all of these things. And our notion is that by having that information, particularly with real-time workloads and consuming it, you can do things like predictive placement of workloads.
[00:25:50] Predictive failure of things tied to operational technology. And, I'm wondering, do you, did you use your own AI to improve [00:26:00] the orchestration algorithm that used to place workloads?
[00:26:04] Simon: [00:26:04] So, yeah. yes, actually, no, not specifically for placement, but for another thing, which is security,
[00:26:12] Matt: [00:26:12] Oh, interesting. Okay. Tell me about that.
[00:26:15] Simon: [00:26:15] security side. You want to look to make sure that swim is operating. In a normal way. So obviously take some definition of normal. But part of the problem is to introspect the running software and ensure that it doesn't.
[00:26:34] Display any obvious signs of having been tempered with. And so it's important. Part of our security architecture in general is to continually introspect to measure and to be confident that we are still executing. Okay.
[00:26:52] Matt: [00:26:52] that's really interesting. That's funny, you're saying, so let's talk about emergent behavior. So when I think of emergent behavior and I want to make sure that [00:27:00] you're thinking it's similarly. Yeah. I think of like, you know, back in high school chemistry where the professor would say, you know, no single.
[00:27:11] Hydrogen molecule is wet, right? No, single water molecule is wet. but when you put a bunch of together, this wetness emerges, is that what you mean by emergent behavior? And can you give us some examples of some of the surprises you've seen? Cause you seem to have delight in the fact that the emergent behavior is surprising.
[00:27:31] Simon: [00:27:31] Well, there are, I mean, okay. There are a ton of really cool things, in the traffic world, you know, just go back to Palo Alto. you know, there are times say pre COVID when say it would be a Stanford football game. What does that mean? Right. So people offer by about these kind of rare occurrences.
[00:27:56] Thanks.
[00:27:56] Matt: [00:27:56] Is there a web agent that attaches to the football schedule
[00:28:00] [00:28:00] Simon: [00:28:00] Oh, you don't need to do that. Actually. It turns out that if you've seen more to them, they're very low probability, but they're in there somewhere. So you can learn. It's like, it's a bit like blueberry muffins. I'm assuming you only have them once in a while. You can know that you like brooding muffins, but you don't have one today.
[00:28:19] Right? So it's there emergent, maybe there's memory in the process, right. Or at least you, when you distill this information, you can come up with quite. Compelling models, even though they're remarkably simple, some way deep in there incorporating this thing is a probability that traffic will behave in a particular way, but you can't predict things.
[00:28:45] You've never seen. That's quite difficult right now, another example in the, Mo provider world, there are some really interesting occurrences, which is that. At night, [00:29:00] everybody goes home and in the morning they go to work, right. And that radically changes the workload on different parts of the network.
[00:29:09] But you can learn that. And that kind of emerges from this vast set of data from independently evolving things. Right. And so you can derive strategies for dealing with effects that occur multiple timescales.
[00:29:26] Matt: [00:29:26] That's it's kind of a, for me, a head exploding idea to that, you know, cause I think of things like, you know, I've built. Business models and economic models and spreadsheets. And so you spent a lot of time, like thinking about what formulas gets you, the prediction of like how much cash you're going to have and things like that.
[00:29:44] And in this case, it's more like, no, you just throw a bunch of little web agents that understand who their neighbors are. And have a bunch of real time inputs and they're doing a little processing on it and you'll come back later and just like, you know, sourdough [00:30:00] starter, this thing has grown into this amazing prediction machine.
[00:30:05] And that just, that's just phenomenal. I mean, I don't even, I'm not even sure how to think about that.
[00:30:11] Simon: [00:30:11] I don't want to make it sound too easy.
[00:30:17] Matt: [00:30:17] Well, yeah, so, so, okay. So let's talk a little bit more about swim. so you, you mentioned open source. what is in the open source? what would I go and what would I find if I went to the open source repo?
[00:30:27] Simon: [00:30:27] what's in open source is everything you need to build a distributed real time application, which does this, everything I've said thus far, are muddle, I guess, a bit. It bears comparison. The model of say data, would they bricks, right? Which is you have these ilities deployability manageability tools for making apps easier to build or introspect, which [00:31:00] approach you.
[00:31:01] Matt: [00:31:01] I see. So in a sense, what I was referring to as the orchestration layer, the magic that I just handed a bunch of web agents and it figures out how to run them and make them all relate to each other. is that correct? Yeah, that's really interesting. and is the open source something that, you're providing because you want your customers to feel like, you know, there's, you know, there's nothing hidden and there's no like.
[00:31:26] Long-term lock-in someone else could build a competitive, orchestration model for it, or are, or, and, or are you looking for contributors to an open source? You're actually looking to create a project. How what's your approach to open source?
[00:31:40] Simon: [00:31:40] So it's in GitHub, right? So there are ripples there. And so as dog, it's a bunch of things. Show we'd love to people to contribute. we bind to the fact that for strategic software, most devs are insisting on open source access. So they want to know something is not going to [00:32:00] go away. maybe it's just, maybe we should call the Oracle consequences. Just aren't going to go to that one again. But, you know, plugins, for example, integration with Kafka Pulser. And so on other brokers that are based in sewn are all open source and need to be added continuously, depending on use cases and customer needs. So it's really. We aren't relying on the fact we aren't relying on the community at the moment.
[00:32:35] We certainly hope that more and more people contribute and take up the software and start to contribute back. And then there are other areas where there are, which are great interest to us, which is the continued evolution of, learn, predict kind of algorithms. Right. Which draw on the experience of [00:33:00] all people like Google and so on and their capabilities, but taken into this new model Lloyd dealing with infinite data and the need to continuously learn and predict.
[00:33:11] And sometimes it's just straight forward analysis, but there is a whole class of algorithms where we want people to collaborate with us, to, to develop and contribute, to swim.
[00:33:24] Matt: [00:33:24] Yeah. And especially coming into this world where, you know, w we're seeing it already, I mean, you're talking about, you know, petabytes of data for one wireless network. Let alone, you know, the three to five that you find the United States that are not even counting the private networks, right? Let alone all the traffic lights, let alone all the street lights, let alone all these devices that are going to become sensorized generating.
[00:33:48] Simon: [00:33:48] There's something really cool there. Okay. So if that's going to happen, then number one, there are not enough data scientists to build all the models to make this stuff [00:34:00] work. Right. they're the scientists, humans to go from build models. Okay. So just think potential use cases, right? So this idea that the big cloud guys come at you with, which is just give us your data and then, you know, ML ops or something, right.
[00:34:21] It's all BS. They just want the data. Right. And, you know, so we, as an industry still have to solve this problem, which is that there are so many sources of data. There is a huge insufficiency of skillset, both in cloud in general, but also in ML. And, so it's easier if we take this approach from the stop, which things can kind of learn for themselves from observation, and then it's the composition of, well, what are all my things?
[00:34:52] I think that's an interesting area.
[00:34:56] Matt: [00:34:56] Yeah. So let's, let's do a little thought exercise. Let's project, let's say [00:35:00] five years into the future. And, let's imagine this world that I see emerging, which is the, you know, the edge is just part of the internet. Right. It's not, it's not this thing we're talking about separately anymore.
[00:35:12] And when you think about like how you build applications, you know, decomposed of services and just services, that there's really a continuum that goes from the device, you know, in your case, you know, some central building and Palo Alto to potentially, you know, a micro data center. The edge of the access network to a regional data center, to a centralized cloud data center.
[00:35:33] And you've got this continuum where you can place storage and compute and workloads and all this things. how does swim, how can swim or not, play across that entire continuum. Tell me how that could work.
[00:35:46] Simon: [00:35:46] Cool beans. So, you know, everybody's in this big, fast about cloud, right? And so if you were trying to distill cloud down to just a couple of things, it will be rests. So [00:36:00] stateless, computing and databases, right? And so risk lets NL server do the work for me and all the state is in a database. And the problem with that model, it's beautiful because it scales nicely.
[00:36:14] The problem with it is that it's a million times slower than the CPU and million, right. That's the difference between hours and milliseconds. So computation today is a million times slower than it should be. God, that's stupid. Okay. All right. When they moved to model. Which is staple. Okay. So, and staple means stuff in memory has state.
[00:36:45] And when Dave shows up, you compute and it's kept in memory, right? So you don't always focus on where's the database and how do I store it's not stored then lies, it's analyzed react. And then if you [00:37:00] have to Seoul the data, by the way, most of the time, because the value of data. It's lifetime is ephemeral.
[00:37:09] You don't store, right? If I can learn on the fly wider about the stone, that the light is green, right? So, this store then analyze approach goes up the window. You go for a staple model where you compute continuously, and then the application layer, maps. The underlying, well, the application needs on to proximity, to data needs for memory compute, other resources like GPS.
[00:37:40] And I think there's a area which is really interesting relative to what you know, your area, which is edge. People always go on about the edge is being close to. So of data and offering low latency. Yeah. Kinda, [00:38:00] but if the thing moves into another edges zone, everything goes, you know, to the wall. Right.
[00:38:07] Or if I choose to store something in some edge location, what happens if I needed somewhere else. Okay. So yeah, you got to move it and that's slow. So. What Sam does is solve the problem dynamically at runtime by moving these staple objects around in memory at runtime and by keeping all compute in memory.
[00:38:34] Right? So everything is in memory. The stateful little objects learn, predict, and analyze their own data. And by Bob the story, right? So for this category of applications, there is really no point in stone, the raw data. And so you can easily integrate with cloud based applications and so on. You can span multiple edge environments and everything is in compute [00:39:00] and memory.
[00:39:01] And you might have terabytes of memory, but you're going a million times faster.
[00:39:07] Matt: [00:39:07] Yeah, that's interesting. and you as a software company, don't care whether your web agents are running on Palo, the city of Palo Alto has computers or Amazons.
[00:39:16] Simon: [00:39:16] Nope. Not at all.
[00:39:18]Matt: [00:39:18] that's
[00:39:18] Simon: [00:39:18] And there is some consequences with through gods and business model, which are kind of cool.
[00:39:26] Matt: [00:39:26] Yeah. Let's talk about the business models. Yeah.
[00:39:28] Simon: [00:39:28] How do we charge for this stuff? So, you know, in my view, mucking up by no bits per second, or by two storage or bits of memory, tough. Cause somebody else wants their piece anyway, right. Sally or an edge provider or a cloud provider or a telco.
[00:39:47] And so no customer don't want to be hit twice. The same thing. And so in swim, we charge only per application layer things. [00:40:00] So really we charge kind on the base of web agents, which are application layer entities, which are learning and prediction from their
[00:40:10] Matt: [00:40:10] I pay by the second or the minute or of how many agents? No.
[00:40:14] Simon: [00:40:14] Number of them, the number of agents that you have per year. Sorry. Right. And then from an enterprise perspective, instead of me charging the same month per incremental web agent, you kind of want a log scale because you want the incremental action to be arbitrarily cheap.
[00:40:36] Matt: [00:40:36] Yes. Right? Yeah. You want
[00:40:39] Simon: [00:40:39] And so we'll do right. and that's working out pretty well for us.
[00:40:46]Matt: [00:40:46] that's really great. That's really great. and. And you want to encourage people to use more because that improves the intelligence of the overall system, which encourages people to continue using this. how do you select the granularity of a web agent? Because, so I'm imagining, okay. A [00:41:00] traffic light it's got, you know, three things that can have steak.
[00:41:03] Well, actually it's got one state red, yellow, which light is
[00:41:06]Simon: [00:41:06] no action in section has probably 80 or a hundred sensors today. Okay. And sex is not a real thing. I mean, in your head, it is. But it's actually a bunch of approaches, a bunch of lights, a bunch of inroad loops, bunch of pedestrian, push buttons and so on. Right? So typically 80 or a hundred sensors per intersection.
[00:41:28] But the intersection is an object which doesn't really exist. But the objects that do exist, all those things I mentioned. Right. So you're going to create these objects on the fly from data. Every time a car crosses a particular loop. I'll see information from that. And then each one of the sensors we'll have, in this case, in the case, traffic, they all have a Latin long will they'll present a latitude longitude.
[00:41:58] And then [00:42:00] the virtual object, the intersection kind of emerges from that. Right. I'm going to join the intersection of the same Latin along as me. All right. And so the link up, right. And I build this graph and then in sections can do simple geo-fencing operation to look around themselves and see things that are a thousand yards away.
[00:42:21] Matt: [00:42:21] And so a web agent, is it one way web agent per sensor?
[00:42:28] Simon: [00:42:28] Yeah.
[00:42:29] Matt: [00:42:29] Okay. Got it. And then this concept of an intersection, as you say, emerges, from a bunch of web agents because of their linkages.
[00:42:37] Simon: [00:42:37] Yes, but also the problem, the main, I mean, really what you're trying to do. So the problem of how do my intersections behave? Not how does red light in this particular lane behave? Right.
[00:42:49] Matt: [00:42:49] And how do you take the prediction that a bunch of web agents are doing that and turn it into. A prediction for the intersection.
[00:42:56] Simon: [00:42:56] Oh, so actually the [00:43:00] intersections are these. I mean, in our case, the, these digital twins of intersections, I guess, continuously solve the following problem, create an input vector from all of this sensor. elements. Right? So every single sensor and all of the sensors that they can see in their vicinity and then forming and perfective from that, through that, through DNA and predict the future values of their own sensors.
[00:43:28] And this could be a red light or a green light or whatever it happened to be, but it could be a pedestrian button and so on. There's an interesting aspect. Which we can go into if you want, which is how'd you do a time. but there are some computational efficiencies you can get there and a whole bunch of other things, but notionally, these agents are continuously predicting like once a second, what the future will be like two minutes up and then they just.
[00:43:56] Matt: [00:43:56] Like you can't just leave that one hanging. What is [00:44:00] it? This time make things difficult. I mean, what is, what were you getting at there?
[00:44:04] Simon: [00:44:04] Oh, I mean, dealing with time is always hard in the sense that a prediction is a future state of a thing. What happens to the next second or second after that? Right. And so the simplest thing, maybe the be stupid and see if it works is to say, well, we predict the future values of this sense of, you know, Oh, well my sensors, what if I'm right.
[00:44:30] Let's assume I am. Right. And they just continue. Right. So each, if you want to look into the future, you basically assume you're right. And just put that in. Is the future values and run the prediction again, right. And run the, run it through DNN again. Now your accuracy decreases as a result because you will be wrong at some point, but it gives you a massively mathematically efficient.
[00:44:57] So other reasons we can go into [00:45:00] way of calculating future values, which are based on the assumption that you were right in the past of being correct. You get great efficiencies that way, right? So that plus sips and convolution allow you to get good bounds on the future. Some way up and in all case, we're competing once a second, you know, 120 steps up.
[00:45:28] No problem. Two minutes with typically within a hundred milliseconds and that's great for costs.
[00:45:34] Matt: [00:45:34] Yeah, that's me. So, so when you think about, you know, the infinite number of use cases that you could apply swim to, and you've obviously done a bunch of them ranging from radio networks to traffic lights, w what application do you see as being super valuable, but that you haven't yet applied it to, like, what excites you about the potential of swim to [00:46:00] help solve?
[00:46:01] Simon: [00:46:01] So, you know, first of all, I think we've seen some things at scale. And we've seen the model works really well. So I'm looking for applications at massive scale, like truly massive scale and, you know, petabytes, but eight is no problem. Okay. So can I imagine them? No. You know, let's say people go from play.
[00:46:28] The cool thing about the tech is this, that it's a million times faster because everything's in memory and. It's typically you need about 10% of the resources that you had to have before. So if you say compared with a big data type architecture in the cloud, you might go from 500 nodes to 50. Okay. So it's really cost effective because we're just taking use of, you know, we're really taking [00:47:00] advantage of CPU, memory pollution at the speed of most law.
[00:47:04]So what's working really well for us is organizations that have got huge infrastructures at scale already. I can see enormous applications in APM application performance measurement, or. You know, ones like yours, where you mentioned you have tons of data, but no, really no real way of drawing insights from that.
[00:47:25] I can see some huge opportunities there. What I really want is people to go and play. I'll give you one last example. Would you kind of cool in a manufacturing facility where we have every pot labeled with RFID tags, and lots of RFID tag is thousands.
[00:47:44] Matt: [00:47:44] The original IOT.
[00:47:46] Simon: [00:47:46] Yeah, right. Tag gets seen by lots of different sensors.
[00:47:51] So think of a digital twin or web edge. And every tech is crepe on the fly millions of no problem. And one [00:48:00] free chip each sensor, whenever a sensor sees a tag, it just basically says, Hey, I saw you. And he was a signal strength. And the obvious thing for a webpage for tags to do a digital twin of the tag needs to compute where it is to triangulate all of the different, base stations.
[00:48:17] Right. And it knows the signal strength. It can learn the attenuation in different parts of a manufacturing facility and so on. And pretty soon you've got tagged items which can tell you that digital presence can tell you where they are. Okay. And then you can say, tell me all the other tags within three minutes, and you can see parts come together to make, you know, a complex item.
[00:48:44] I can airplane. It's pretty awesome stuff. By the way. Here's the big fail. That's this, that system runs on two, two raspberry pies. And so the learning is if I can build something which runs on a raspberry PI, [00:49:00] I can't charge you a million bucks.
[00:49:06] Matt: [00:49:06] And okay. And what's the solution. Put the raspberry PI in this rig really big box with a lot of blinking lights.
[00:49:19] Simon: [00:49:19] Yeah, well spent the Oracle logo on it.
[00:49:23] Matt: [00:49:23] Yeah. Oh my gosh. I Simon, I have a couple of last questions. It has been an amazing conversation. so, so w what is the, if you could, you know, look out, let's say 18 to 24 months. And think about, you know, there's cause start startup companies. Like we're always at the mercy of like everybody else's pace, right?
[00:49:51] Like customer adoption, technology rollout, all of this stuff. w what are, if you could go into the future and kind of nudge, you know, [00:50:00] tip over some of the dominoes in this path towards like Nirvana for swim AI, w what would you push on? What would you accelerate?
[00:50:08] Simon: [00:50:08] we've been on tour through IOT. And one of the things that's been tough, there has been that although a lot of organizations are trying to get to. And your street photo or something else, they, they fight a battle, which is that further out, do you get the closest to the real edge? People are interested in vertical outcomes or they want a better oil rig, but they don't want all the different pieces in there.
[00:50:36] Want to come make it. Maybe they didn't have the skillset to go and build it. So. Getting there for a lot of orgs is aspirational, but not easy. And so we found there that customers tend to want to buy solutions. So if you look at say C3 or AI, right, they're a solution company they'll come in and solve your problem.
[00:50:57]and they're less [00:51:00] interested in building, Sushan from software components. So what we are doing is. Working with orgs like carriers or people who have the skillset to build. Then there it is their business. They just have assets, a huge scale already, and they are magical for us. So if I had a, a magical spelled will be to move the broader set of industry participants faster down the path of gaining the benefits of IOT.
[00:51:33] Matt: [00:51:33] That's really interesting. You know, there's a, a common trope in the container, in container, you know, like Docker the container world, also with. With like a continuous deployment and continuous integration. And the idea is that the thing that's hard are the cultural changes, not the technology, a lot of the technology exists.
[00:51:54] It's just the culture changes. And it sounds like you're finding something similar. Right? I mean, I mean, I can imagine if I'm used to [00:52:00] consuming, you know, petabytes of data and pushing it through my Kafka spark Hadoop, you know, whatnot. It's. I mean, you're going to get me to throw out my data by prying it out of my call.
[00:52:15] Good hands. I mean, it must be really hard culturally to get to the point where like, no, you don't just rely on this little web agent to predict.
[00:52:22] Simon: [00:52:22] Good example, there here's a great example of few. we did failure prediction for a manufacture, which was assembling printed circuit boards. And the learning is this. If you call Freddy back from lunch and the machine is not about to fail, you're in deep trouble. And yet learning and prediction is fundamentally a statistical thing and you will.
[00:52:49] Matt: [00:52:49] Yeah. It's going to fail sometimes. And that's how it learns.
[00:52:52] Simon: [00:52:52] Yeah. So, and get for this manufacturer, you know, things failing and breaking was part of the cost [00:53:00] of doing business. So it was all competed in. So when Johnny got pissed off that you had called them back from lunch and the machine wasn't about to break. He was ready to throw you out. Right. And that was hot even though on average or whatever you are better.
[00:53:19] It's just hot.
[00:53:20] Matt: [00:53:20] The law of unintended consequences. Well, it's Simon. So, so, people can read about more about swim.ai at swim.ai. if people want to get ahold of you online, what's the best way to do that.
[00:53:31] Simon: [00:53:31] Oh, I'm Simon. It's Swim.AI . Yeah.
[00:53:34] Matt: [00:53:34] Wonderful Simon, this has been an absolute pleasure. thank you for coming on and rerecording this interview with us and, I hope you have a great day.
[00:53:41] Simon: [00:53:41] Thanks so much.
[00:53:43] Matt: [00:53:43] Thanks Simon. I really appreciate you doing this again. I it's, this is a really interesting topic. Have you guys,