How can we create a “digital nervous system” and how will this change the edge? In this conversation, Bill speaks with David Sprinzen, VP of Marketing at Vantiq, an application development platform for real-time edge-native applications. David and Bill discuss how modern technology can pull inspiration from the human nervous system, and dive into the future of generative AI at the edge.
How can we create a “digital nervous system” and how will this change the edge? In this conversation, Bill speaks with David Sprinzen, the VP of Marketing at Vantiq, an application development platform for real-time edge-native applications. David and Bill discuss how modern technology can pull inspiration from the human nervous system, and dive into the future of generative AI at the edge. They also talk about data privacy, hallucinations, and small language models.
“In the new world what we're looking at with technology is effectively creating a digital nervous system. That means, how do you take data in from sensory inputs from cameras, from sensors, from IOT devices, from humans, and how do you make sense of it? How do you react to it? How do you store it?”
“Data is not value in and of itself, data is a mechanism by which you derive value.”
--------
Timestamps:
(01:20) How David got started in tech
(07:37) What drives people to the edge?
(12:08) What does edge native mean to David
(20:30) What does naturally distributed mean?
(22:11) How Vantiq thinks about “low code”
(28:33) Data management needs more attention
(30:40) The role of generative AI
(33:42) The future of small language models at the edge
(36:08) AI at the edge in healthcare
(40:08) Digital twin city project
(42:26) How does Vantiq identify the best projects?
--------
Over the Edge is brought to you by Dell Technologies to unlock the potential of your infrastructure with edge solutions. From hardware and software to data and operations, across your entire multi-cloud environment, we’re here to help you simplify your edge so you can generate more value. Learn more by visiting dell.com/edge for more information or click on the link in the show notes.
--------
Over the Edge is hosted by Bill Pfeifer, and was created by Matt Trifiro and Ian Faison. Executive producers are Matt Trifiro, Ian Faison, Jon Libbey and Kyle Rusca. The show producer is Erin Stenhouse. The audio engineer is Brian Thomas. Additional production support from Elisabeth Plutko.
--------
Follow Bill on LinkedIn
Connect with David Sprinzen on LinkedIn
[00:00:00]
Narrator 1: Hello and welcome to Over the Edge. This episode features an interview between Bill Pfeiffer and David Sprinson, the VP of Marketing at Vatiq, an application development platform for real time edge native applications. David and Bill discuss creating a digital nervous system and how modern technology can pull inspiration from the human nervous system.
They also dive into the future of generative AI at the edge, discussing federated learning and small language models at the edge. Over the Edge. But before we get into it, here's a brief word from our sponsors.
Narrator 2: Over the Edge is brought to you by Dell Technologies to unlock the potential of your infrastructure with edge solutions, from hardware and software to data and operations across your entire multi-cloud environment.We're here to help you simplify your edge so that you can generate more value. Learn more by visiting dell.com/edge for more information, or click on the link in the show notes.
Narrator 1: And [00:01:00] now, please enjoy this interview between Bill Pifer and David Sprintzen, the VP of Marketing at Vantiq.
Bill: David, thanks so much for joining us today. This should be a really fun conversation. I'm just really looking forward to hearing your perspective on things. Uh, yeah.
David: Happy to be here today, Bill.
Bill: So we'll start with our standard question, which kind of goes, takes us way back. How did you get started in technology? How did you get here?
David: Well, I've always been extremely interested in technology.
I actually came from a background in electrical engineering and computer science and then took kind of a circuitous route through neuroscience and studying the biological systems and how they encode information, how they pull Insights out of, in, you know, in the human system, it's, it's our senses. It's the, you know, the original sensor data.
And so how does a biological system actually take that information in and convert it into insights and actions and [00:02:00] reactions and reflexes? And that got me deep into kind of the data science of how a biological system. And then I ended up joining Vantiq in a lot of ways because, you know, There was a huge overlap in what we're doing with technology and how that is similar to the human nervous system.
So, in the new world, what we're looking at with technology is effectively creating a digital nervous system. That means, how do you, Take data in from sensory, you know, inputs from cameras, from sensors, from IOT devices, from humans, and how do you make sense of it? How do you react on it? How do you store it?
That concept is very near and dear to my heart. So I was pretty excited to find a company that's on the front lines of figuring out how to do those kind of digital nervous system type applications.
Bill: Wow. That's, that's a heck of a background. I don't think I've gotten like mentally derailed so early in a podcast before, but that just [00:03:00] takes me down like a whole bunch of rat holes.
You know, in the nineties, AI was all about expert systems. You, you just build rules and you keep building rules until eventually boom, you get sentience that didn't turn out to work. And so we started to mimic human biology and animal biology and things like that. But now we're getting so many inventions that are.
mimicking, you know, cool animal features and the way they fold and the way they unfold and the way the muscles work and whatnot. And I mean, applying that, that neuro, the neurological and the biomechanical intelligence to systems of IOT, in addition to the processing, not just the AI, but the actual, you know, data collection and things like that.
That's, that's a really cool perspective. I love it.
David: There's a, Interesting angle that I think people are starting to understand or at least acknowledge, which is the way we've built our digital systems and software and technology over the past few decades has been very database [00:04:00] oriented and there is a huge role in the database, you have to store data, but if your entire model is data storage, then you're really missing it.
Interesting. A big part of, it's not a matter of storing data, it's a matter of acting on data, and not, the data is in fact not the important asset, it's what is the value, what is the action that needs to be taken, and that's how the biological nervous systems work, they don't just store things, if you touch a hot stove, you don't think about, you know, How do I store that information and then maybe I should put my hand away?
There's a reflex and that's where we need to start thinking about our digital systems and AI in particular. So that's where this world of real time distributed AI systems starts really coming to life and you start looking at the world through how do you convert data as just an ephemeral object that you're looking for insights, actions, and value that you can derive in that moment.[00:05:00]
Bill: And the data management is such a, I don't think it's given enough priority in the whole process, right? We're used to thinking in terms of data centers and clouds, where the amount of data is relatively small relative to what we're going to see at the edge. And so, That idea of immediately process and then decide, do I need to write this?
How long do I need to store it for? Is it valuable to begin with or is it just process and dump? And I, there's going to be so much more process and dump, just react immediately in the moment and then move on because, you know, there were no surprises there. It wasn't a learning moment. You didn't touch a stove.
If you touch a stove, that's a learning moment. You really have to keep that, you know, capture that data into long term storage. If you pick up a pencil, there's no learning.
David: And the long term storage is an after the fact decision, right? There's a whole bunch of things that you do in the moment. Then there should [00:06:00] be a data architecture and intelligence and an AI architecture that's supporting that in the moment understanding and decision making.
And then after the fact, you decide, you know, how are we storing this? Is there a data lake? Is there long term storage? Or is this stuff that we can just throw away? Yep.
Bill: When I was building data centers, it was all about just capture everything and store it forever. And it wasn't that big, so it didn't really matter.
And you know, that was, that was no problem. We just had an archive system and we dumped everything there and we had a big pool of data that doesn't really work now with all the distributed stuff that we're getting and all the high volume stuff that we're getting. Anyway, I came across a sentence when I was looking into Vantiq and it just, it caught my attention.
Vantiq is a low code edge native app development platform for naturally distributed applications using edge data. There's so much there. That's like a whole conversation. And I'm going to keep coming back to this sentence if that's okay. And we can just kind of unpack that piece by piece [00:07:00] because there's so much in there that I want to talk about.
I love that sentence. I don't know who wrote it, but hats off to them. It's, it's just fantastic.
David: I can, I can probably take credit for that one.
Bill: So first off, when customers come to you for a project, do they ask for the edge? Are they asking you for help with IOT? Are they asking you for help with a business outcome?
What do, what do customers look for when they come to you for a project?
David: Yeah. Well, we don't often see them coming with the first thing is, Hey, we want the edge. And now we're going to figure out, you know, what we're going to use it for. Generally, it's because of, you know, there's, I would say, three main driving forces behind the move to the edge.
One, of course, is latency, right? And that's the performance and the ability to do that real time element, moving compute as close to The source of the data is possible. So I think a lot of times we're talking to people that have real [00:08:00] time problems and they're trying to figure out how to address them.
They recognize that the edge is a core piece of infrastructure and architectural model is going to enable them to address some of the real time needs. The second part of it is one of data privacy. Right? And so we often talk to companies where they were never, for example, healthcare, right? Healthcare, there's so much limitations with where the data can be stored, how you're using it, who has access to it.
And it was never a simple, you know, store everything in one location, because there's elements of, you know, personalized data that you were never going to centralize. And so the healthcare industry is a good example of, of, you know, they're trying to build. Digital solutions and services that move out into the edge where it can be close to private data stores.
So, for example, in the hospital or even in a person's home or in a nursing home, being able to [00:09:00] move digital applications close to where the data is located so that you don't have to deal with any increase in, in privacy concerns. People can hold, you know, have, Full kind of access and custody of their data.
And then the third element, which I think is maybe the most compelling, and I think it's over time the one that is going to be the most important for the edge, is the explosion of the amount of data. And If you go back to what we started the conversation with, which is data is not value in and of itself, data is a mechanism by which you derive value, and the purpose of the edge is an area where you can apply it.
Bring, you know, analysis and, you know, thresholding and conversion of something that's raw, high volume data into maybe, you know, statistics, or if nothing else, looking for situations of interest, then you can actually start handling that [00:10:00] explosion in the amount and the heterogeneity of data that we're starting to witness.
So I think that that over time is going to be the main driving force of why people are even interested in the edge. Let's be able to handle the explosion of the amount and the types of data by throwing most of it away and just being able to capture key moments or key events of interest.
Bill: So when I, I use those same rough three things, I call them speed, scale, and security, rolls off the tongue easier, but it sounds like customers are coming, looking more for some sort of a result, right?
I want to get value from this data, but it's too. Fast. I want to get value from this data. It's too big. It's too private.
David: And generally it's about, we recognize that there are things that technology and AI can do to make us much more operationally efficient and then digitally enabled. But to do those things, it, we need to be able to have [00:11:00] the, the, the, the infrastructure to do real time understanding and awareness.
And a big thing that Vantage is doing, because we're not obviously the only company that's looking at real time analytics, but one of our, our key focuses is on the ability to, to create, and in a low code platform, Really rapidly and easily create the, the models in the way in which you're converting raw data into something that's actionable.
And then I think the, the thing that oftentimes people don't take that next step, which is, okay, you now have a, you know, situational awareness, you now know what's happening. How do you actually convert that into action? How do you integrate those insights into, Business workflows and automation cycles and collaboration with humans.
So we're, we have a number of tools that say, okay, when you detect something now, here's how that's actually embedded into workflows, into operational procedures that are integrated into the business itself. Got it.
Bill: Okay. [00:12:00] So jumping back to my sentence. Vantage has a low code, edge native application development platform.
What does edge native mean to you? I've started to see, I mean, I looked at it a couple years ago. There wasn't really much momentum. Now there is some momentum, but I don't think there's yet anything close to agreement of
David: Yeah, in my mind, I mean, edge native at the end of the day is built to run on the edge, right?
Quite literally. So what does that mean? Well, in a lot of cases, it's being able to build the applications that, uh, That embody the key values that the edge brings. So real time, scalable, secure. One of the things that we on the Vantix side have found is that a core tenet of being edge native in our mind is the ability to use a asynchronous data ingestion environment.
So usually that's going to be using an event driven architecture where everything is done in With Pub Sub loosely [00:13:00] coupled messaging, because then you're removing a lot of the dependencies between how different systems are communicating and you're able to get the scale of a lot of asynchronous data coming in at once and being able to handle that.
So in our mind, there's a event driven architecture is kind of a core tenet of being edge native.
Bill: That makes sense. So back to your point about, it's not so much about the database as the data. So you're not running like a credit card processing system that has to be a hundred percent consistent a hundred percent of the time.
You've got a series of loosely coupled events that are kind of distributed all over the place. And if they're loosely synchronized, that's good enough for most applications.
David: Yeah. And, and I mean, we're, we're still doing like, you know, stateful processing on the edge. It's not that there's no local storage, but that's not for the purpose of, of longterm.
That's for the purpose of trend analysis or looking at different streams [00:14:00] and trying to understand what they mean when you look at them in combination.
Bill: Makes sense. Makes sense. Okay. And what sort of differences do you typically see in customers that are deploying, building, deploying, architecting edge applications versus cloud or data center applications?
And fundamental to that, do you see that? Do you see customers building things for the edge or is it a cloud component and an edge component? Or is it an app that may be deployed in the cloud, may be deployed on the edge? Is there, you know, are there people building edge specific apps that are destined for
David: the edge?
Yeah. A lot of the groups we're working with on the ISV side, on the large company side, we're working with telco companies and we're working with, you know, energy and utilities companies with healthcare companies. A lot of them are understanding the need for edge as being a core part of what they're developing.
I think right now there's still a sense of, you know, there's going to be, [00:15:00] Elements of this that are going to run on the cloud. There's going to be elements of an application that are going to run on the edge. And it's, that's not even a binary decision because the edge isn't just one thing, right? There's a huge spectrum of what the edge needs.
And we have projects, for example, where there's the far device edge. which is, you know, the thing that's going to maybe do that first pass of data processing. Then there's regional data centers that are going to take that from any number of edge devices and do some of the convergence and the combination.
And then maybe you have cloud it's then, you know, the, the, at the end of the day where a lot of this stuff is being stored. And so you might have, Very over time, an increasing hierarchy of, of different processing locations, each one of which are doing different parts, different pieces of the puzzle. So it's not necessarily just, you know, either or cloud or edge.
And there's a number of considerations that go into what to [00:16:00] run where, of course, there's cost considerations, there's data considerations, there's privacy considerations, going back to some of the tenets of why to use the edge in the first place. So In a lot of ways, what we're working on with companies is, okay, let's talk about what your application does holistically.
Now let's take that big picture. Let's partition that into the fundamental building blocks that compose that application. And now we can determine, based on those building blocks, what should be running where. In fact, we have a number of patents around being able to do that. Intelligent using AI to partition the application into those modular pieces and then automatically deploy them based on what makes the most sense given their infrastructure and the requirements of the app.
Well, that's cool.
Bill: So you're, you're using AI to help your customers Automatically do kind of tiered processing, what has to go out to the edge, what should be kind of long term storage in the cloud, [00:17:00] whatever, and regional in between those things.
David: Exactly. Oh, that's kind of cool. The whole concept of distributed applications, so it's not just cloud or just edge, it's distributed everywhere.
Take care. And you're talking about very heterogeneous locations that are doing slightly different things within that application framework. You need to create abstraction layers to be able to handle that level of complexity. So what we're doing on our side is making it where the application logic is one layer, and then the way it's actually deployed.
Now, it's being done in drag and drop low code tools that automatically take a lot of the burden of figuring out how to manage the deployment, how to optimize the deployment for given outcomes, and then, of course, you have things like fault tolerance, if an edge node goes down, if you have loss of connectivity, how do you make this system work in degraded environments or degraded situations?
So, There's a lot that goes on under the hood to support what we call [00:18:00] mission critical applications on the edge.
Bill: Operating in degraded conditions is going to be one of the, one of the challenges of the edge that we don't really have with the cloud so much. It's pretty much up or down with the cloud and you know, you pretty much have either access to near infinite resources or nothing.
And at the edge, it's going to be a whole lot more of We're supposed to have three servers, but we only have two. We're supposed to run at 60%, but we just spiked at 99. Whatever. And deciding which applications take precedence and whether it can, you know, which applications can run degraded or can push to a different location.
That's going to be, there's going to be a lot of magic there. That's going to be an interesting challenge to solve. So I would imagine that we're going to need some platforms that have some intelligence in the backend to help, to help write those rules. That's not going to be simple.
David: That's right. We're doing some applications on, for example, on autonomous vehicles at sea.
And they have satellite connectivity, but that satellite [00:19:00] connectivity is not always present. So you need the application to be constantly running on that device. piece of equipment, doing whatever it's supposed to be doing. And then the moment connectivity is present, then you might have some kind of back and forth communication with the cloud, the, but being able to run headless, uh, autonomously on the edge, that's a core part.
And most of the, the more complicated edge applications out there. The edge isn't just, you know, a server sitting in a factory. Of course that can be one thing, but a lot of the more complicated ones, the edge is. You know, a very dynamic piece of equipment that is moving in space and time doing different things and might, you know, actually be involved in the action, so to speak.
So of course that's like healthcare. We're doing stuff, for example, in ambulances, an edge device in an ambulance doing patient monitoring and then interfacing with doctors in the hospital, but. It only, it [00:20:00] only interfaces with doctors when there's connectivity and there's services that it can be done locally as well.
Bill: That makes sense. Makes sense. So back to, back to my sentence again, you said something about distributed applications in there too. And in that sentence, you called out naturally distributed applications. What does naturally distributed mean?
David: Well, I guess there's two parts to that. One is why would you distribute an application?
So I think that's a matter of, you know, because you have. elements of the application that you need to be running close to the data sources. And then you're going to be running that not just in one location, but maybe a number of locations. So usually that would be the reason why you're distributing it.
And then to make it distributable, you need to have modularity and partitioning. So naturally distributed applications means that there's both a compelling reason to move those workloads out to the edge, and The way it's [00:21:00] structured, which is a lot of what we're doing, is making it naturally partitioned so that it knows how to structure the application where these are the pieces that go out.
Bill: That's an interesting point. The naturally distributed piece, I guess, centralizing like we have for our entire careers so far, wasn't really a natural choice. It was driven by the requirements of the technology. We had technology in the centralized places, so we centralized stuff because that's what we had.
And now it's pushing back out again, which is more natural because that's where the data is and that's where we want to use the applications. You know, your mobile phone, you're not standing inside a data center to use your mobile phone. You'll walk around places. That's the point of mobile. So yeah. Okay.
Naturally distributed because it doesn't have to be centralized now. I just like that sentence even more now. That's just so much fun. Okay. So Vanity has a low code edge native application [00:22:00] development platform. We haven't talked about low code and I haven't talked much about low code. What drives customers to build their applications?
Low code versus traditional methodologies.
David: Yeah, it's a really good question. And we have a kind of a funny relationship with the wording of low code because in the industry, oftentimes when people hear low code, they think of Platforms that are catering to non technical professionals, like citizen developers, who are, you know, going to take a stab at trying to build something that now they can even, you know, imagine trying to develop because the, they don't need to be gurus in the coding language.
So that's a lot of the industry's interpretation of the word. What we're doing is more about being able to create abstractions so that. The systems that you're building is not so complicated that it's effectively [00:23:00] untenable to build and manage. And so a lot of what we're doing, it's not, it's, it's for the purpose of agility and productivity, but it's so that you can tackle.
These very comprehensive distributed applications that without some levels of abstractions being able to look at the logic, the building blocks, the way it's being partitioned, what the application is doing, the way in which it's being deployed, and where, and being able to literally drag pieces of the application to different environments and create the rules by which it's being deployed and dynamically migrated depending on those rules.
All of that. If being able to go to low code tooling, often graphical interfaces to look at these things visually and understand how they're working. That's, A requirement for these kinds of applications to even be possible. So we've worked with a number of companies where they've gone through, you know, multi year development cycles with huge teams trying to build [00:24:00] edge solutions and.
aren't getting anywhere. The system is not getting the results they want, it's not scalable, and it's way too complicated. And then we're coming in and saying, look, use our platform, we'll work with you. And in a matter of a couple of weeks or months, you now have everything you've been trying to do because you're, you're not focused on that.
The, the complicated infrastructure, the, the, you know, the, the foundational elements underneath the hood to make this thing run. You're just focused on what you're building, what the business logic is and where it should be running.
Bill: Okay. So kind of making some simplifying assumptions, building some of the mechanical parts so that it's easy to call functions, as opposed to having to build each one and figure them out.
Like the AI that's helping to distribute the applications more intelligently based on where it should go so that people don't have to code that into each application that they build individually. And so not so much about a [00:25:00] lack of sophistication in coding, which is, I mean, that's the first thing on my mind too, right?
Like now I can start building applications and I'm not a coder, ha ha ha. But not necessarily that, just having packaged functions that are available with simplifying assumptions made that you can bypass, I'm sure if you really want to get your, get your hands dirty, but shortcuts more than anything
David: else.
And the name of the game there is agility, right? And there's agility across the agility to build these things. There's agility in the ability to deploy them, especially when you're talking about distributed environments. How do you make it easy to deploy it, but also being able to And then the third piece of it is agility.
Agility to, to change it on the fly or to change it as the situation evolves, as you want to add new capabilities and technology, maybe you're switching out sensors, maybe you're switching out the AI, those [00:26:00] three types of agility make it so that now So, You're not having to be structured into one very brittle plan from the gecko.
Mm hmm.
Bill: So again, you know, just trying to get this, get my head in the right shape for this. How does that affect A move to DevOps and modern software practices. Again, I tend to think of DevOps as like we code everything from scratch. It's all in Python and we just do it. So just, you know, we're going to sit in this back room and we're going to make it happen.
That's DevOps in my world. And it doesn't go together with low code and, you know, graphical interfaces, but that's me and I'm not in that space. So I suspect I'm very wrong, but how does. I mean, is that orthogonal to the conversation? Like you go low code or traditional and you go DevOps or you don't in modern software practices.
Are they connected together? What are you seeing from [00:27:00] customers in that space?
David: Well, I think agility and, you know, agile practices are very much a part of the DevOps methodology. And, and so we're looking at, at, you know, A lot of what we're doing is working with DevOps teams to then say, Hey, instead of using the tools you're used to, focus on these higher productivity tools.
And then in, for example, a two week sprint, you're not just trying to get one part of it done. You're maybe taking on a larger piece of it, like the full development of the MVP, Application. And then the second sprint, you're looking at, okay, how we're going to start testing the rollout of this. And then the third sprint, you're already running in the, on the edge and doing load testing and performance testing.
So being able to look at those sprints as tackling bigger parts of the puzzle. By using high productivity environments, [00:28:00] we find it goes hand in hand.
Bill: Okay. All right. Last time, coming back to the sentence, I'm going to read it again. Phantic is a low code, edge native app development platform for naturally distributed applications using edge data.
And the last piece of it, I said, I said earlier, I don't think data management has given enough attention, but identifying what to process in the first place versus just delete. What's high value, what's process and dump, what's, you know, store forever because it's super valuable data. To do that, With edge data, where it's distributed all over the place and you're making these, you know, each site has to make its own decisions and it's all streaming data.
So it's constant. You can't go back and reprocess the data because you just don't have time. There's new stuff coming in 24 hours a day. How are you seeing customers? Figure out how to make those decisions, let alone automating all of that stuff in terms of, you know, [00:29:00] this, this constant stream of data with new stuff coming online all the time.
David: Yeah. It's a, it's a good question. And I don't think that there's a simple answer. I think it's a number of different things. Of course. You can do like basic, you know, a data analysis on the edge looking for, you know, crossing a certain threshold where the data is showing something that's suggestive of a problem, or you could use unclassified anomaly detection algorithms to look for any You know, piece of data coming in that is surprising for whatever reason, and then you can try to figure out what that means.
There's a number of rules, pretty much all applications, there's a number of things that they know they need to work with, right? They're like, immediately, okay, here are the things we know if we can detect this, you know, this crossing of threshold or when these conditions are met. We know that this is something that we need to deal with.
So there's a number of things that kind of people bring immediately they just are aware of. Where it gets a little trickier is when you're [00:30:00] trying to look for patterns or look for trends, look for situations that haven't been hard programmed because maybe you didn't know to look for those kinds of things, right?
And so interestingly, we're Looking at generative AI as having a pretty meaningful role here as well. And why generative AI and not traditional AI? Of course we're using traditional AI as well, but generative AI has a kind of a unique and somewhat nifty ability To take, you know, input. Let's say we have a sensor reading coming off of a piece of equipment and it's not something that we have seen before.
We can use generative AI to take that data in and automatically look at the documentation of that piece of equipment, automatically look at maybe the stored history of that device and start trying to analyze what could be going on here. [00:31:00] So even if we have a hard programmed. It, the system to look for a specific problem is if it's documented somewhere, we can store that in what's called vector database and then generative AI can look at the vector database in real time as that data is coming in and start giving advice or recommendations or even taking action so that the ability for it to handle non deterministic style Situations where it's not been hard coded to know exactly what to do, but it's able to kind of learn on the fly.
That's something that generative AI is kind of opening the doors for. So now we're seeing that as a huge area of innovation where all of a sudden, of course, you're going to have traditional AI models that are still looking for things that you kind of know to look for, but You're able to now create systems that can learn on the fly as things are happening and use other pieces of information like your knowledge [00:32:00] stores and your documentation and even, you know, company policy and interface with humans.
to try to problem solve and do real time decision making collaboratively.
Bill: That's a really interesting use case. I've, I've been looking for realistic, interesting, practical uses for Gen AI at the edge. And it's, you know, relatively slow, relatively very power hungry and expensive and limited, but you know, it's, it's large language models.
So it can tell you, What kind of wine goes best with fish? And is that part out of spec? And people are like, oh, we're going to use generative AI to tell if our parts are in spec. No, no, that's, that's craziness. But at the same time, Once you identify an outlier, going back to the documentation and analyzing the situation and maybe looking at a digital twin to try and figure out why it's an outlier, that's a really good use case for it.
And because it's, it should be, you know, one off corner cases, [00:33:00] oddities, outliers, being a little bit slower, being a little more expensive, would be fine because the next thing you have to do is pull in a person and we are much slower than generative AI and much more expensive than generative AI. So that's, that's a pretty cool use case.
I want to watch that more. That's a really great idea.
David: There's a part of that too, that's really important to highlight, which is You have, you know, large language models, which are trained on huge amounts of data. You have smaller language models or private language models that are more specific to a given environment, whether, you know, it's a smaller amount of training data and it's a smaller model footprint, but both of those, you don't want to just rely on the training data.
You want to be able to have those, being able to call upon real time information and knowledge stores that. Maybe it wasn't trained on, but it can access. So that process is generally called retrieval augmented generation or RAG. And that's becoming, I think, a foundational [00:34:00] pillar of how these systems are going to be developed.
Because now you can take these language models and Even if it's not trained on something, you can store, like, you know, policy, like let's say documentation on a piece of equipment. You store that in a vector database. The moment something happens, the language model, it doesn't need to know about that piece of equipment.
It just needs to know where the documentation is. And it can then read the right pieces of that documentation, learn, In the moment of what's going on and give advice, even if it hasn't been explicitly trained on that. So that's a, that's called in context learning. That, in my mind, is where this whole industry is going to go.
Because now, all of a sudden, you're not having to worry about crazy amounts of custom training and ongoing continuous training of these models. You just need to make sure you have the right tools. Uh, vector database with the knowledge stores needed for it to answer a given question and understand the situation.
Right.
Bill: Especially as we get into [00:35:00] the good generation of small language models, as opposed to large language models, you don't need that kind of flexibility to answer a specific question that has, you know, specific gates around it. It came from this piece of equipment. You need to use. The documentation that comes with the equipment and understanding the industry that you're in and not people, people talk about AI hallucinations because it's been trained on everything and it's seen everything.
So who knows what the hallucination is going to be? It could be anything, but if it's much more gated, then you'll have much smaller hallucinations and much less frequent, I would imagine. So small language models with rag would be. A really, really cool implementation against that use case that would be much more efficient and much more practical.
Super cool. I love it.
David: And the hallucination thing, a big part of that is you have, when you ask a, you know, chat to question, they'll give you an answer. You don't know how it came to that answer. When you're starting to use retrieval augmented generation, it can actually tell you, this is the information I [00:36:00] use to get to this answer.
And here's the pointer to this documentation. This is, this is how I'm referencing why I'm giving you the answer I'm giving you. So it's, it's much more trustworthy and then it avoids potential hallucinations. One other point to add here too, is what we're doing with a lot of these applications is by moving them out to the edge, you can have.
You know, for each environment, for each location, the data storage, the, the, the layer that's supporting the generative AI can be very specific to that location. So, a good example of this is in healthcare, right? You have patient data that's very sensitive. And if you wanted to make, let's say, a digital nurse that's able to talk to the patient and understand what's happening, give them advice, maybe even intervene if there's a potential problem, you want that to be running locally and interfacing with that patient data right there.
You don't want that to be something that you're centralizing because then you run into all the privacy concerns. So the [00:37:00] ability for it to be learning. You know, effectively a federated, distributed fashion where now every location is running its own version of the same application, but it's specific to the data of that environment.
Bill: It feels like healthcare is going to drive a lot of the really interesting advances in this space. Because there's so much money in it, there's so much need. It's health, you know, what wouldn't you pay to have your health protected a little bit more? So, you know, there's a lot of money to play with, but boy, the regulations, the restrictions, like as soon as it leaks someone's data, it's all over.
So that's got to be just super, super locked down and you can't make that happen. guesses about medications. So it has to be right. So that's going to be like the precision that you have to do, but then the resources that you have to play with. It's, it's going to be a fascinating space to watch as soon as, as soon as a couple companies can figure out how to handle the restrictions.
Cause [00:38:00] I don't know, but that's a space that scares me off.
David: This goes to a larger Challenge in the AI community, right? Which is around regulation and governance and especially with generative AI, where it's not necessarily bounded in what it can say and do. So you have a lot of companies going, you know, how do I even govern a system where I don't know exactly what it's going to say?
Bill: Right. And it's talking to patients about sensitive healthcare issues. Hmm. Okay. Don't be insensitive ever, but you know, go ahead and be automated. Good luck.
David: Yeah. Maybe not. Yeah. But this is where a lot of the art of AI application design comes into play because there is absolutely going to be people that figure this stuff out.
And you know, our, our role and on the Vantage side, we don't want to figure it out. We want to make it easy for people to innovate and experiment and figure out what to do. And so [00:39:00] that goes back to the idea of agility, right? You, you need to be able to. Build these systems in a timeframe where now you're constantly refining them, you're testing it out, you're figuring out what's working without it being these massive development cycles.
So if you can speed up the feedback loop by which you're experimenting and innovating, then all of a sudden you can start figuring out exactly how to make these systems functional for a given application.
Bill: Yep. Makes sense. So I would imagine you've worked across a whole swath of different kinds of clients, different kinds of projects, different, different industries.
What's been your favorite project so far? What was the most fun to work on that stands out?
David: Okay. You know, there's, there's a number that come to mind. Let's, I'll, I'll focus on one because I think it's, it's kind of interesting. So there was a big digital twin project for a city that we worked [00:40:00] on here in the U S.
And they were doing cameras inside of the alleyway to be able to look for vehicles, the location of vehicles. And actually, the way that they paid for the whole thing was because waste management services, so literally the trucks that came to pick up trash and recycling, were getting stuck in alleyways that were blocked.
And it was charging literally 2 million a year in overage fees. to the waste management services because once the trucks are blocked, they have to wait for a tow truck to come and move the car. So they were able to build a digital twin of the alleys and use computer vision on the edge to basically just get a sense of where the cars are and if the alleys are blocked and interface that into the waste management services.
Now, It was, you know, almost kind of a trivial application, but now all of a sudden we have a digital, a real time digital twin of these alleys, we can start looking at all the [00:41:00] other city benefits. So there was a number of things now around, you know, public safety and security, around being able to do, you know, Pedestrian safety, being able to look at homelessness, being able to support other city services beyond just waste management.
All of that now can interface into that same system because at the end of the day they were just trying to help garbage truck drivers know. Which way to go.
Bill: That's amazing. I love it. It's, it's one of those use cases that there's just nothing sexy about it, but then I'm sure it, you know, cost justified itself in no time flat because it's so practical.
Very cool. Okay. So when you start a project, what do the best, the most effective customers say or do or know that from the start you think to yourself, this project's going to be really smooth and it's going to come out awesome. How do you identify, how do you identify those [00:42:00] projects that have the best chance of success?
What characteristics do they
David: have? Well, I'll tell you what characteristics they don't have. Which is, they're not holding on to the idea of the first thing you do with L data is store it, right? And, and we run into this a lot where even we'll, we'll help people understand what to build and architect it, but then they still go to the default paradigm of, okay, the first thing I do is store everything.
And, and if they, if that's what they do, then we really almost need to hit them over the head multiple times saying, no, stop thinking about it in terms of strict data structure. You have to think about it in terms of. Deriving insights, action, and then storage, maybe if it's even necessary, is going to be the final step.
So I think, you know, that's kind of a qualifying thing that we're always looking for is, are they, are they starting to think about these systems and are they understanding the need for, you know, data being an ephemeral entity that's coming through that you're [00:43:00] driving value from and that action and the orchestration of response to something is, is where the real.
The other thing that we're often looking for is for, for our projects, is having lots of different data already present that they're not deriving value from. So of course there's a lot of projects where they're going, okay, we need to install new things, new cameras, new sensors, et cetera. Mm-Hmm, . But a lot of times they already have that, right?
They have, they have a lot of data that they're just not effectively using. And so that's often the most fun for us is when we can say, look, have you thought about if you, you know, look for these kinds of incidents and are looking across, you know, this, these sensors and these cameras and, you know, the location of these humans, if you combine all of that and start being able to do real time edge intelligence against it, These are the kind of things that you can start looking at and automating.
Then it gets really exciting because they go, Oh, I've always wanted to do that. I've always, I always knew that there was something here [00:44:00] that we could do, but I could never figure out how.
Bill: That makes sense. Yep. And especially, I like the idea of, of doing more with the stuff that they already have. Right. We, we came across an example a while back of a city that had four, five, six cameras.
on every intersection pointing in the same place. And one was doing license plate detection, and one was doing, you know, traffic violation detection, and one was doing something else. And can't you just share data across them? Like install a better camera once and just do it, process it multiple times. And they went, Oh, okay.
Like just the, the extra expense. And I'm sure it was, you know, each one was done by a different subcontractor on a different contract, the silos that we love to talk about. But yeah, using, using the stuff that you already have is, is a much more cost effective way. And I bet that lets you show a much bigger win, much faster for a much better ROI because you don't have to install all those expensive things.
You just use [00:45:00] the processing that you have. Super cool. So, David, how can people find you online and keep up with you? All the great work
David: that you're doing. Yeah. We have a website, vantic. com. It's v a n t i q. com. So anyone who's interested in learning more, welcome to go to that website. We have a lot of material, you know, thought leadership around where the edge is going, the kinds of applications you can build for the edge.
And then we also have a developer community, and if you're interested, we have a community portal, community. fantech. com, and we're working with a number of companies on helping them develop solutions and ideate, and then experiment with what can be done for the edge. So, If you're someone who's interested in trying to learn what can be built and how to get started, we would be very interested in working with you on that and actually giving you a sandbox environment to start playing [00:46:00] around.
Bill: Fantastic. Love it. David, thank you so much for the time today and for sharing your great perspective. I know I learned a fair bit. I hope all of our listeners enjoyed it and learned some stuff as well.
David: Thanks, Bill. Great conversation.
Narrator: That does it for this episode of Over the Edge. If you're enjoying the show, please leave a rating and a review, and tell a friend.
Over the Edge is made possible through the generous sponsorship of our partners at Dell Technologies. Simplify your edge so you can generate more value. Learn more by visiting dell. com slash edge.