In this episode, we dive into the current state of AI and its future ramifications for society with Ed Henry, Distinguished Engineer - Artificial Intelligence at Dell and an expert with over 18 years in the tech industry. In a conversation that weaves between science and philosophy, Bill and Ed discuss the challenges of model drift, the societal impacts of technological "feudalism," and the implications of AI trying to recreate human intelligence.
In this episode, we dive into the current state of AI and its future ramifications for society with Ed Henry, Distinguished Engineer - Artificial Intelligence at Dell and an expert with over 18 years in the tech industry. In a conversation that weaves between science and philosophy, Bill and Ed discuss the challenges of model drift, the societal impacts of technological "feudalism," and the implications of AI trying to recreate human intelligence.
---------
“Should people get involved in AI? Yes, it's going to become a technology that is not going away. Pandora's box has been opened and we're not closing it anytime soon. I do believe that write once, run once code is coming.”
“ It used to be that technology was a differentiator for a company. It's not anymore; it's a necessity. What you build on top of that technology that becomes a differentiator.”
--------
Timestamps:
(01:47) How Ed got started in tech
(04:46) Understanding AI model drift
(08:51) Transformers and future AI applications
(18:58) Technological feudalism and social implications
(33:24) The philosophical pursuit of Artificial General Intelligence
(34:12) The role of AI in automation
(48:44) The limitations of current AI models
--------
Edge solutions are unlocking data-driven insights for leading organizations. With Dell Technologies, you can capitalize on your edge by leveraging the broadest portfolio of purpose-built edge hardware, software and services. Leverage AI where you need it; simplify your edge; and protect your edge to generate competitive advantage within your industry. Capitalize on your edge today with Dell Technologies.
--------
Over the Edge is hosted by Bill Pfeifer, and was created by Matt Trifiro and Ian Faison. Executive producers are Matt Trifiro, Ian Faison, Jon Libbey and Kyle Rusca. The show producer is Erin Stenhouse. The audio engineer is Brian Thomas. Additional production support from Elisabeth Plutko.
--------
Follow Ed on LinkedIn
Follow Bill on LinkedIn
Producer: [00:00:00] Hello and welcome to Over the Edge. This episode features an interview between Bill Pfeiffer and Ed Henry, a distinguished engineer at Dell Technologies who's focused on artificial intelligence and an expert with over 18 years in tech. In a conversation that weaves between science and philosophy, Bill and Ed discuss the current state of AI and how it'll change our world moving forward.
They cover the challenges of model drift. the societal impacts of technological feudalism, and the implications of AI trying to recreate human intelligence. But before we get into it, here's a brief word from our sponsor.
Ad read: Edge solutions are unlocking data driven insights for leading organizations. With Dell Technologies, you can capitalize on your edge by leveraging the broadest portfolio of purpose built edge hardware, software, and services.
Leverage AI where you need it. Simplify your edge. And protect your edge to generate competitive [00:01:00] advantage within your industry. Capitalize on your edge today with Dell Technologies.
Producer: And now, please enjoy this interview between Bill Pfeiffer and Ed Henry, Distinguished Engineer for Artificial Intelligence at Dell.
Bill: Ed, thanks so much for joining us today. This is going to be a fantastic conversation. Digging into AI and maybe a little bit of edge we'll see, but mostly just talking about tech. I'm looking forward to this.
Ed: It's going to be great. Thank you for having me. Hopefully we can head down some interesting paths that get people thinking about this technology more than just like how cool it is, but like, how is it going to impact us?
And what is it going to do to society and technology as a whole? So I'm excited.
Bill: That says we're going to run long. So just getting us started. How did you get started in technology? What brought you here?
Ed: Wow. Well, you know, I was just reflecting with one of my mentees the other day that my career is old enough to vote this year, which is a little [00:02:00] bit interesting to me, thank you.
But I've been in industry now for 18 years and my first job, believe it or not, was actually working on the geek squad back at Best Buy. So I was fresh out of high school and I needed a job and I went to Best Buy and I've always been interested in computers in general, and I'd always fixed my friend's computers and whatnot in school, so I decided to take a job at Best Buy.
And wearing the little tie and the help and fix, you know, personal computers. And, and I've kind of naturally grown my career from there. So I got a job at a local hospital after that, where I worked help desk and troubleshooting where I also did desktop engineering there as well. If that's even a thing anymore, I don't know if it is.
And then from there, I just kind of moved in the ops space until about 2012. I actually got a job in academia, so I ended up working at the University of Connecticut building something called the Connecticut Education Network, which is a couple hundred miles. Actually, I think a couple thousand miles of dark fiber all over the state of Connecticut, providing connectivity to K 12 institutions, fire [00:03:00] stations, police stations, municipality buildings, basically providing internet connectivity to everybody in the state.
And that's where I got Connecticut. exposed to AI and machine learning actually, because this was during the time of like software defined networking and science DMZs were a thing and how can we accelerate data exchange between different research institutions and whatnot. So I was able to kind of leapfrog from the world of operations and running data centers and running networks to how do we do new and interesting things with these resources.
So from there, I ended up working at a startup. So I was at the University of Connecticut for, I don't know, about two years, two and a half years. Built some interesting software defined networking solutions, won a hackathon actually, which I'm pretty proud of back in like 2013, 2014 for what is called software defined WAN now.
Bill: Okay. Yeah.
Ed: Right. So, so we built a little using a software defined networking controller where we could tag traffic by doing reverse DNS queries. So like when you would like press update on your [00:04:00] iPhone, we would check where that particular IP address was resolving to and we would route your traffic accordingly because back then when everybody pushed update on their cell phone on a corporate Wi Fi environment, it crushed the network.
Right? So we wanted to put policies in place there. So I ended up working for an SDN startup called Plexi. I was there for. Quite a while, leapfrogged through industry a few times. And I've been here at Dell for about five years, six years now. I don't know. I'm losing track. And I work on anything and everything AI related from robotics to computer vision to time series forecasting.
A lot of my work has been Gen AI recently for obvious reasons. But yeah, that's, it's been a pretty eclectic history within the industry. But like I said, just the other day, I was thinking, man, my, my career can vote. That's a little weird to think about.
Bill: So let's jump into. Let's call it the current state of AI.
As enterprises plan for AI, you build your model, you build it, you train it until it has a certain level of accuracy. Then you send it out into the world [00:05:00] to do its thing. Over time, models drift. They get less accurate. The world changes. The compute environment changes. Whatever, stuff changes. Because it's the world.
The ideal is you get to a certain level of decreased accuracy and you retrain the model. But as I understand it, AI is changing so fast that almost nobody actually retrains their models. They basically rebuild them because it's moving so fast that there's a whole different paradigm. And so you just start from scratch because it's so much better or smaller or faster or whatever.
Is it really moving that fast? And how do people keep up with it?
Ed: So there's two ways. So to answer your question around drift, yes, that's a natural state of reality. Like, you're not going to be able to move away from that. A model is a representation of reality. And if reality changes, which it does, your model needs to change underneath as well, right?
Physics has been trying to solve this problem for years. Like the standard model doesn't describe all of reality. Maybe, maybe not. So we try to find these [00:06:00] models, which are some representation of reality. So in the case of like, what is the The most popular technology in AI right now, generative AI, the way you see this reflected in the technology that's shipping today is like cutoff dates for training for models, right?
So if you were to go and pick llama three, one that was released by meta recently off the shelf, you would notice that the model, the 8 billion parameter model, if I remember correctly, the cutoff date for training on that is December of 2023. So what you can assume is that the pre training regime for that model contains no new net new information.
With respect to what has been produced between that delta of December and now, right? So that does mean that we have to retrain the model in some way, shape or form. Now, there's two ways in which you can go about retraining that model. Obviously, continue with the pre training process, right? So add new data to your data set, continue the pre training process, and off you go.
You can also do something called fine tuning, which really. Depending on how big your data [00:07:00] set is, if you're integrating it into your original data set, could be just considered a pre training continuation. But the reality becomes you do have to update the model in some way, right? So we do see that happening, but for a few different reasons, not just because the world has changed around us, right?
Mostly because we want to specialize the model in a specific task. So it isn't just a temporal constraint that you need to train the model to do something different. Relative to what the model was trained for, but then that, that retraining problem or that time progressing problem just kind of compounds itself because then you're going to have to continue the training of your original foundation model and then retrain or continue fine tuning your model again.
So it kind of becomes this iterative process. Right? So I think that, you know, from the perspective of engineering. We're, we're, it's pretty well understood for some value of understood. Like we can't tell you why the model is making the decisions it's making on output, but that's okay. These are connectionist models.
They've been black [00:08:00] boxes from the beginning. We're learning a little bit. Like if you look at the work going on inside of Anthropic and OpenAI and Google right now, or monosemanticity, which is this idea of like, what are the semantics contained within this particular model? And what is the concept in the model?
And how do we identify if the model has a sufficient. Representation of a concept, right? So if anybody who's read like the anthropic paper, they, they're able to clamp the clamp, the residual streams on some of their models and like convince the model is the golden gate bridge. So like, it'll answer as if it is the persona of the golden gate bridge, which I think is really interesting because that means we're touching on something fundamental here in the properties of the models, right?
So that's the engineering side. So it's pretty well understood that we're going to need to, you know, continue the training process for these things because the world has changed. That's not going to go away. That's something we've understood in machine learning and AI for a long time. Where I think we're running out of a bit of steam is what the transformer is itself, which is the [00:09:00] architecture has unlocked the ability for us to be able to do interesting things.
There's huge automation pushes all across many different industries right now, but the transformer architecture and the inductive biases that exist inside of that model, which is usually a choice that you make that somehow captures a property of reality that you want. Baked into the model. Auto regression is a good example for transformers.
Another is like co occurrence exploitation. So the fact that words or tokens tend to co occur, that's one of the things exploited from an information theoretic perspective inside of these models. That's getting a little. A little long in the tooth, right? So you're noticing a lot of companies right now, and you're hearing a lot in the media about how we need more data or companies are putting together these giant factories of synthetic data set generation processes, whether it's in the form of using other large language models to do it or using humans to then generate and label data sets for you.
The fact of the matter is. We have struck something that will enable [00:10:00] us to do some new and interesting automation. But from the research side of things, there's a lot of work to do with respect to building whatever the next generation model is going to be. That said, Bill, to answer your question directly, when that new vein of gold is found, everybody will jump to it.
Right. And there's a reason. And the reason is because we want to make sure that we are somehow exploiting the the overall performance characteristics of these models relative to the use cases that we have. What I think the last thing I'll say about this, what I think is interesting is. While transformers seem to be running out of steam, sort of, for some value of running out of steam.
I say that with a big asterisk, by the way, because I'm sure a lot of people beat me up about that. Applied to the text domain. But again, these models are sufficiently general enough, this is why they're called foundation models, that they can be applied to different Domains, different topics and different concepts, right?
So you'll probably see a lot of interesting exploration and transformers in the world of time series forecasting [00:11:00] again here over the next couple of years. And the reason why is because the sequence links that we're talking about, you know, 128 K tokens on an open source model off the shelf 128 K, depending on what that window of time is, what you map that back to in terms of representation of time.
That's a pretty long window of time. So you'll be able to predict with some accuracy, whatever, over some length of time, whatever the next output on a sequence should be. And when you look at like where this is getting applied. Tesla has been doing this for a long time inside of the self driving systems that they have inside of their cars.
There was a paper released by wave, which is a self driving car company with a model called Gaia and Gaia uses transformers to simulate versions of reality. It's a generative model, so what the model is doing on the fly as it's driving down the road is generating sequences and trying to determine what what is going to happen in the next step, given the set of sequences that it's generated, which I think is really cool, because what ends up happening is if you have a self [00:12:00] driving car driving down the road, they have a great video of this, actually, and there's a car comes into view of the camera for the vehicle because it's using a generative model under the hood.
It can simulate a few different realities, one of them being the car pulling out completely in front of you.
Bill: So it's actually running what if scenarios while it's analyzing what is. Right,
Ed: right. So you can generate different sequences and make sure, and if there's structure contained within that sequence, like I talked about, the car pulling out, the car didn't actually pull out.
But you can use that simulated representation of reality for planning. Right. And planning, it comes down to, should I press the break or not? Right. So there's, there's a lot of planning that comes into this as well. So when it comes to these foundation models, we'll see them applied and we already are, but in more force across a lot of different industries that aren't just natural language processing over the next couple of decades or.
Decades. Like I know anything the next couple of years, right? The next couple of days. Let's go with that. That's the safest bet. These days. That'll
Bill: be enough. It's [00:13:00] moving so fast. So that was that was a whole lot more philosophy on top of the AI I was expecting. What is a concept?
Ed: What
Bill: does the AI believe its concept?
is. Oh yeah, that's, that's pretty meta. And now we've got Gaia that's starting to, for lack of a better word, dream, right? Imagine what's going to come and start performing risk rewards based on that stuff. And wow. Okay. Interesting, interesting spaces. Wow. And the transformer models running out of steam, but at the same time, the state of the art is so far ahead of the state of adoption.
That even if the state of the art slowed down for a while, that would probably be okay. Cause it would stabilize for a bit and let enterprises actually catch up and start implementing some of this stuff. And even if we use, you know, AI from six months ago, which is not state of the art anymore, like that would still be amazing in industry.
That's wow. Okay.
Ed: And I think that's what you're seeing though, right? Like to your point, the adoption of this [00:14:00] technology is non trivial. Right. And I think it's, it's more than just because the tech is complex, right? The tech stack is sufficiently complex and it takes some time to learn, but I think what we're going to notice over the next couple of years specific for enterprises is.
You know, I've noticed a pattern over the last, I talked about my career being 18 years old. Let's talk about patterns that has identified as a glorified pattern matching machine. It used to be that technology was a differentiator for a company. It's not anymore. It's an, it's a necessity. And that is what you build on top of that technology that becomes.
A differentiator.
Bill: Yep. I remember when companies considered their network to be a differentiator. Now it's just plumbing. And of course you have one on who cares
Ed: exactly, exactly. So this adoption of the latest generation of AI technology, so we'll call it gen AI. We're going to see a similar effect where the businesses are going to have to shift their processes a little bit to account for and accommodate for this new technology.
And that takes [00:15:00] time, right? That takes time for the information to disseminate within the organization. That takes time for the amorphous blob that is most enterprises to then shift and head in a particular direction related to these technologies.
Bill: We also need time to build up trust.
Ed: Exactly.
Bill: People have to trust the model.
They have to trust what comes out of the model. They have to trust that the model isn't stealing their data. So, you know, the, the number of companies that don't want their people using Gen AI tools. Oh, you might leak data. I think that's just a fundamental trust thing right there.
Ed: Yeah. And it's a valid one.
I mean, if you look at this, this big battles going on right now between all the major. Tech companies, because they've effectively been stealing data from each other, like I'm going to pull down all these YouTube videos and I'm going to use that to train a model that's kind of violating the service agreement that you have by having a YouTube account, right?
And the same thing applies with we saw this with Reddit, right? Reddit pulled the plug thing. Thank the Lord that they're able to do or whoever that they're able to do that [00:16:00] because like that there's treasure troves. We all go to Reddit because the answers that we need are there. So that's an obvious hunting ground for these folks who are going out there and trying to find new data sets that are related to the problems that they're going after.
And frankly, I think it's, it's drawing this kind of weird line between what is intellectual property. What is a digital representation of myself? Right. Because as a person who's posted on Reddit before, and then my answer becomes part of a training data set that's used for one of these large companies that have a hold on the market, how do I, as the one who produced that information and posted it on the Reddit side of the world, just that again, as an example, I'm not picking on Reddit specifically, but how do I exercise control over that?
Bill: Yep.
Ed: Right. So I think that to your point, it's just, it's an adoption cycle and it just takes time. What I think is interesting is how fast the. Large, the companies who are good at this technology are outstripping the ones that aren't, and that's what worries me the most truthfully, because I tend to [00:17:00] try to pay attention to the transfer of power as opposed to the, the interesting technology tidbits.
That's interesting. Don't get me wrong, but it's the transfer of power that really kind of makes me step back and go, okay, so how do I participate in this system in a way that I feel like my fundamental freedoms and representation of ed are protected in a certain way? You know what I mean?
Bill: Yep. So, so let's dive head first into that one.
We've been talking, we've been talking, hearing, observing, right? The whole Moore's law thing. Doubling every compute power, doubling every 18 months is slowing down and, you know, we get, we get quantum decoherence in the two to three nanometer model. So there are just physics, right? You're wearing a shirt that says because physics.
So because physics, that's why, you know, we're, we're having a problem, keep getting faster. So now we're looking at specialized chips and different architectures and things like that. So. Microsoft, Amazon, all sorts of other companies [00:18:00] are starting to build their own, not just processors, but AI chips, AI processing chips and processing stacks.
And they're building them to differentiate their AI capabilities, which is very cool. Gives you more efficiency, gives you more speed at less power, which is fantastic because we have to reduce our power consumption somehow in this exploding AI that's sucking so much electricity, but. But now we have potentially even less workload portability, right?
I'm using this company's AI that runs on their chips in their data center, and I have to pay them for every bit that I use. The technology implications of that, we could probably talk about for a good while, but what are the social implications of that?
Ed: Yeah, that is a wonderful question. So I kind of like went off on a path when you mentioned something.
So I think what we're headed in my head. So I'm trying to formulate my [00:19:00] response where I think we're headed is almost like a technology, technological feudalism, right? And what I mean by this is you have the surf class. Those are the plebeians or the plebeians, right? And those are the folks that kind of assume that they're going to have a home on this particular set of land because the vassals have talked to them and convinced them that they should live there and work toward the person who owns the land, which are the landowners, right?
So I think that this technology is kind of flying in the face of the social structure that we call capitalism. Right. And what I mean by this, and I'll get to the point, but what I mean by this is like, when you look at the, the, the person in a capitalist system, the person who has the power, the ruling class is the person who owns the means of production.
But there is no means of production in these services and tools. It's a platform. It's a land that you've been given to then build upon [00:20:00] or work in some way, shape or form. Right? So we see businesses cropping up using all of these tools and technologies that these companies are building and they skim off the top and say that I'm going to take my percentage off the top and they design an algorithm to maximize that percentage they get to take off the top.
Right. So it seems and looks and it feels like we're headed down a path of almost technological feudalism to me. So what does this mean from the perspective of us as people utilizing these technologies? We need to be well aware and well versed in at least the implications of the technology. If you are unaware of AI in general or what it is doing to the markets that we participate in right now, you should become aware of that as quickly as possible.
So much so that like I give my kids like a weekly or monthly rant about. Like pay attention to this. You're entering the workforce in the next couple of years. These are the things that you need to be paying attention to. These are the patterns that I'm seeing. And this is how the technology is being used to exploit you effectively.
Right? So I think that when it comes to. How do we participate [00:21:00] in this socio technical system? It comes down to making sure that we design constructs in society that allow us to keep control, right? So when you look at like GDPR over in the European Union, while that does arrest technological progress, and a lot of people have heartburn over that because how could we possibly slow down the beast?
We're supposed to keep going. I think it actually empowers folks to be able to have control over their new representation of themselves. Right. So when I talked mathematical models before, right, when I talked a model being a fundamental description of reality, there is a model of all of us on all of these platforms, and we should be able to affect and control where and how we interact with this model and what representation a company is able to collect from us and have the power to exercise whether or not a company can or should have access to that.
Right. So I think that what's going on right now is an aggregation of power. And like I talked about previously, it's that aggregation of power that worries me the [00:22:00] most from a social perspective. So what I try to do again is to just do things like this. This is the current state of the technology. Here's the things that the leading indicators you should look for.
And the giving the technology, what I'm happy about is the fact that open source is what's been powering AI for the last. Probably 40 to 50 years at this point. And I hope that doesn't stop anytime soon for two reasons. The first is that allows folks like you and I, Bill, to go and reason about what the state of the art is to even have a pass at understanding what these things are.
And the second is if we can somehow understand the open source software. And then the business models that folks are wrapping around this, we can kind of head off this natural feudalism that happens, right? So, and this is all my opinion, take it for what it is, but I do see a transfer of power going on right now and an aggregation of power that worries me, frankly.
Bill: So it's an interesting point. The [00:23:00] digital feudalism, right? The concept of this is a platform rather than just a piece of software. The piece that kind of tweaks me is in the feudal society that was based around land. There is so much land. You literally cannot create more. You have, you know, you have a space.
It has boundaries. But when we go digital, you can create as much as you want. You can create many platforms. And so in theory. Other people could come in and create a better platform. Now that had just placed an order for I think 300, 000 NVIDIA GPUs, which costs like 40 billion. Not many people can afford to do that.
But at the same time, how long Are those going to be the state of the art? NVIDIA is going to be releasing new GPUs every year. They've committed to this. And I think AMD and Intel have committed to the same thing in the spirit of competition. Like, no, we'll [00:24:00] keep up. And so everyone's accelerating the pace of their technological development.
And so last year's GPUs are not selling at a premium. It's just this year's GPUs that are impossible to get. So next year's GPUs are going to be amazing. Can you afford to spend $40 billion to build an equivalent platform? I don't know. Can face, can meta afford to spend another $40 billion to build its own, you know, to maintain that platform in sda, the amount of resource burn that we're gonna be going through is gonna be unreal.
But then also the data burn. Right? Right. We're seeing, you know, we're running out of data on the internet. Good heavens is YouTube not even keeping up with it. Holy cow. But then there was an image generator that was generating images that had Getty watermarks on them. Oops. Well, cause it had trained on so many things that were watermarked that it just like, Hey, that's what makes a good image.
It's got [00:25:00] this watermark on it. Ah, oops. How do we maintain privacy through all that?
Ed: That is a great question. So, the, we'll, we'll approach this, if that's okay, from the perspective first of the custom hardware, because we didn't get to that on the last round. I apologize. And then we'll, we'll go at it from the perspective of how do I guarantee, or how do I at least approach the idea of guaranteeing some form of privacy.
So we've known in technology for a really long time that custom chips for custom operations are going to be the most energy efficient, right? And I use energy as the baseline because that's what we're trying to minimize as much as possible, whether that energy is in the form of heat dissipation or in the form of power consumption, it's energy.
At its, at its core, right? So you mentioned physics before. So, how do we optimize a set of operations as much as possible in the face of that? Well, you cut custom chipsets. Now, the good news is, we see this kind of parallel in biology all the time. So like you and I walk around all day, but we don't think about [00:26:00] making my heartbeat.
I don't think about having to breathe. I can, I can, I can assume control of that system, but I'm not necessarily consciously in control of it all the time. A lot of that autonomic functionality has been pushed down to a certain portion of your brain. So you have a custom chip. For some value of chip, a lot of people, I'll get screamed out by neuroscientists for making the parallel of brain and processor.
We'll go down that path later on, but you have this custom chip for some value for lack of a better term in your brain that handles all of that autonomic operation and handles it really efficiently, right? So, so efficient that, like all of your brain on average, I think only consumes about 40 watts of energy, which is crazy to me.
So we see this, this, this kind of play out in biology where a custom piece of computational hardware comes to fruition because it's a necessity. Right? And we're seeing that right now with respect to how can we exploit GPUs or these big triangle machines. To perform the algebraic computation that we need [00:27:00] in order to train the models that we're going after.
And then even above and beyond that, how do we get them deployed out to the edge? So a lot of companies are spending a lot of money right now building really efficient technology stacks to do that. Because to your point, Bill, you were mentioning, how do I have control over myself and my representation in this system?
From the technology company's perspective, it's how do I control this entire thing? Thing, right? And how do I define constraints around this entire thing to make sure that this widget will operate within these bounds at all given times, the only way you can guarantee that is through cutting custom silicon, right?
So for a long time, we got drunk and happy on x 86 x 86 solved all the problems. You can write any type of computational workload and run it on x 86. And as long as it compiles down to assembly, it'll run. And then GPUs happened, well GPUs happened a long time before this, but somebody got the interesting idea to use GPUs for what they are, which is giant matrix multiplication machines, and turns [00:28:00] out that maps really well to things like deep neural networks.
So 2012 happened and everybody got super excited. So what we're seeing right now is a technology investment in building these custom chipsets in order to build an optimization and an efficiency into the entire tech stack. But. This comes at the cost of what you were talking about, which is like, I'm going to adopt, say Google, cause I have a.
I have an Android smartphone, so I'm effectively married to the Google ecosystem and the Google tech stack, right? So how in the face of all this, do I guarantee my representation is respected? Uh, that is a non technical issue. That is a legal issue. That is an ethical and moral issue. And I think that including folks in the process.
Of designing these technologies more than just what a lot of these labs talk about with respect to including people and what is called like either the red teaming or the purple teaming process for for [00:29:00] cleaning up data sets that are used for training models or validating outputs of the models and making sure that the models are kept on some sort of between the some sort of lines on the highway in terms of like providing guardrails for data curation, I think we need to you.
Push companies as a people who consume these technologies to build better tools for us to intuit about how this process works inside of their system. So when Ed uploads a photo to Google Photos, I should be able to understand what systems that photo hits and I should be able to understand what the overall impact of that photo living within the Google ecosystem looks like.
Now, that's a very high level statement that does not include a lot of implementation detail. The reason why I do that is because I don't think we quite understand what we've done. where we have machines that are able to pick us and our picture and our likeness out of millions of images in, you know, fractions of a minute.
And that is, that is an incredibly powerful technology [00:30:00] that most technology is inherently inert when it comes to having some sort of negative or positive connotation, but as soon as a human who isn't picks it up and does something with it, then that's cause for concern. Right. So I've, as, as folks who consume this technology, I will always push people to please put pressure on these tech companies that you're consuming services from to somehow give you some visibility into what your representation looks like on their platform, because back to your ecosystem comment, they own you the second that you type or push anything into this device, you are in their ecosystem, whether or not you want to be.
Bill: Sure. Yeah. Well, and that may point again toward the goodness that could come from having the transformer model slow down and start to hit its limits. We need to find what the next thing is going to be. But that lets us slow down and consolidate your point about GDPR, slowing down tech advancement in Europe, slowing down to humanize [00:31:00] things, not a bad idea, slowing down so that we can figure out what should be legislated, what needs to be legislated.
We want to do that minimally so that we're not all over the place and, you know, just have massive overhead. But there are some things that really kind of need to be centrally controlled by the government rather than just a, a multinational corporation. Otherwise we get to this, you know, so many neo futuristic shows are like the corporations have taken over and governments just don't do anything anymore.
We don't really want to get there because of AI. So it'll be interesting to see where we go with that now. Jumping tracks to a different thing. I'm just curious if you have any thoughts on this, but up until like the nineties or 2000, the state of AI, the fad with AI, if you will, was more toward expert systems.
Right. We'll write so many rules into the AI, it will eventually just somehow become self aware and become intelligent. And then it followed the rules because, you know, it was written to [00:32:00] follow the rules. Duh. And so then we started to go with more neurological models and that's where we are right now with deep learning, right?
Mimicking neurons and multi layered complex models that are trying to mimic the human brain. There's another model coming up. I mean, it's a fascinating point that the human brain pulls about 40 Watts and like the latest rack from NVIDIA, I think is 130 kilowatts, just for like rack, something like that.
It's, it's unwritten. Maybe it's 45. And the hyperscalers are building to 130. Anyway, a whole lot more than 40 watts. A whole lot more. Yeah, so is there a different path that we need to explore before we can get to the next level of efficiency, right? We've gotten to effectiveness. Now we need to get to efficiency with effectiveness.
Do you [00:33:00] think that's going to be a different model or do you think we just keep diving into this model? That's a whole lot of theory right there and a guess.
Ed: No, no, no. It's a, it's a great question. And the reason why I'm hesitating to answer is because I think the root of your question is in whether or not we feel satisfied that we've sufficiently replicated ourselves.
And I don't know if we'll ever get there. So like AI has been this kind of constant chase outside of what we see in business today, but from a purely academic or philosophical perspective, we've been trying to recreate ourselves since, I mean, Hephaestus was creating automatons in ancient Greek mythology.
Right. So like, we've been trying to recreate ourselves for millennia. And I don't see that stopping anytime soon. So to answer your question very directly, I don't think that transformers are going to be the end all be all. I do think that there are other ideas [00:34:00] and things that we are going to need to pursue in order to get to this thing called we'll say it AGI, but in all reality, I think that the technology that we see today is really just going to become a.
Boon for automation, right? Like if we, and then I'll get to the other side, but when we think about what natural language and a natural language interface does to being able to use computers, that is the. Mode of information transfer that we have been using for millennia at a minimum. So you and I utter utter noises at each other and we somehow reconstruct that back into words inside of our head.
Right? So being able to build a layer of natural language on top of all of the systems that we've built that allow us to interface with those are going to be incredibly interesting and it will be a huge automation boom for the. Other side of this. We are not going to stop marching toward recreating ourselves.
I don't think anytime [00:35:00] soon and we are woefully misinformed. If we think that a single model like a transformer somehow captures the nature in the essence of what it is to be human, because for a couple of different reasons, one, we have no idea what intelligence is, let alone what consciousness is.
There's early work, not early at this point, I guess, probably a couple decades old of like, How does consciousness interface with the brain, right? Like when you get a tooth pain and that pain, we can track it by measuring the signals in your head and all of the traffic patterns inside of the neurons in your head, but then that stops, where does the pain go?
Like that's the next set of questions that we have when you feel the pain, we can track it to your brain, we can see all of the different electrochemical interactions going on, and then all of a sudden it's gone, but you still feel pain, right? So what is that interface? And that is where we are going to continue looking with respect to where is AI research headed?
[00:36:00] Right, but the good news is, you know, using this mathematical modeling approach still brings interesting results to the table for solving real problems that exist in the world. So, like, transformers are really cool, but Sepp Hochreiter, who's the lead author on the LSTM paper, which is another autoregressive approach to modeling sequences, just released a paper called XLSTM, which is an extension to the LSTM idea, and it is outperforming transformers in certain areas.
Right. So there is this idea of a no free lunch theorem in machine learning and this idea of a no free. There's a few of them. But the main idea behind a no free, no free lunch theorem or concept in general is that no one model works best for all data sets. Right? So we are going to continue designing AI systems for different problems.
And those specific AI systems are going to be form fit to that problem. Right? So we may have struck gold in natural language with generative AI, but that probably won't solve more [00:37:00] complex scenarios where you're trying to integrate things like vision. And language, and we see multimodal models and they're able to do new and interesting things, but to your expert systems comment, how do you then layer the rules that map back to being a human in reality on top of those technologies?
And we don't have those answers right now. And I think that we are going to continue heading down the path of exploring other ideas. When you look at folks like Yann LeCun, who's the. He won the Turing Award. I mean, it's like the Nobel prize for, for computing. And he's even saying that like, transformers are cool, but they're not the answer, right.
That we're all interested in. So I think that over the next, you know, probably a couple of years, we'll actually see the academic. Circles start to be like, okay, you know, we deliver transformers to industry. We've really kind of pushed this technology out to the people, right. And it's time for us to go and go back to the drawing board and say, okay, we haven't solved this clearly.
There's no humanoid walking around that I can have a, an [00:38:00] existential relationship with. Right, but there is a way in which we are automating certain portions of society and reality that are useful to people and I appeal to Steven Boyd. He's a, he's a professor at the Stanford University as a book on convex optimization is actually called convex optimization.
But on page 17 or something, he calls out the difference between science, a scientific discovery and a piece of technology. And the difference is people can use a piece of technology without necessarily caring about the details underneath. And I think that we've hit that with transformers. Like people are able to pull a model off the shelf, feed some data into it, get a response out of it.
And they don't necessarily care about the inner workings of the model. I think we are fully in technology mode when it comes to these foundation.
Bill: And I didn't miss the tide of philosophy there. I asked if we were, if we were stuck on the transformer model and you said humans don't want to be alone in the universe.
Yeah, it is also [00:39:00] true and is an interesting way to look at it. It's not even this light is bit wrong. That just, that's a really interesting way to look at it. And yeah, it's kind of. An effort to replicate ourselves and find ourselves in the machinery that we're building and okay. Yeah. It's especially interesting because there's so much talk about digital assistance, AI based assistance.
And now. Microsoft, Google, Apple are all pushing down LLMs to mobile devices so that we can have better AI based digital assistants out in the world, right with us, like our little tiny children that we carry in our pockets. And they're going to become more and more capable and run more and more stuff.
Yeah. It ends up, that's just going to disturb me every time I think about my phone now, like, okay. Yeah.
Ed: It's a, it's a weird representation of me. Yeah.
Bill: Yeah. [00:40:00] Yeah. And then I just put it in my pocket and walk around with it and oops, I forgot to charge it and it died. Oh no, it died. It's okay. It'll be back tomorrow.
Yeah.
Ed: Yeah. I think, you know, it's, this is what I mean when I mentioned earlier, like. We need to be able to articulate this to folks on a grander scale than just burn and churn.
Bill: Like I
Ed: get it. We gotta, we gotta keep producing in order for people to consume and that keeps the system going, but eventually, we need to reason about what these technologies are in a way that impacts us on a deeper level than just our pocket.
To use your analogy. So like that keeps filling my pocket with money. But that money eventually doesn't mean much in the face of eventually I don't exist anymore. So what does it mean for us to build a technology in the face of not existing? And when you look at like early AI research in this space, like Jeff Hinton, anybody who's familiar with Jeff [00:41:00] Hinton, really well known, won the Turing Award as well.
He likened the idea of what we call Embeddings today in the world of large language models as thought vectors. So this idea of creating an embedding of the language that Bill uses every day and capturing the patterns of the language that you use, I can then use that embedding to reason about who Bill was.
What patterns bill used in his speech, what topics bill used to talk about. So he, he called these things thought vectors. You probably won't hear that a lot anymore because AI hype, but he did, he called these things thought vectors because I think he was heading down a similar philosophical route, which is like, how do I make sure that I persist even after this meat bag dies is what it becomes, you know?
Sure.
Bill: Yeah. So time is marching on as it does. Let me get a little bit more tactical here. I was going to ask. [00:42:00] How do people get involved in AI if they're not already, but should people get involved in AI if they're not? I mean, it was it was the big sexy thing and it was going to be, you know, the massive six figure paycheck and every company was going to need an AI engineer as we get on toward commoditizing and.
Well, commoditizing, right? Does every company need a data scientist? Are there going to be, you know, you go to Azure or, or AWS or Google GCP or something? And they have models. NVIDIA has models. Intel has models. AMD has model, right? Everybody has pre trained models. At what point does it become more about NVIDIA?
Data engineering or automation and application engineering to leverage the AI models that are there? Or is it still going to be about custom creation of AI models to do just the specific thing that needs doing?
Ed: Ooh, that's a great question. [00:43:00]
Bill: Should people get an AI? Is that going to be a big booming business or is it going to be that you need to, there's going to be something else about how you use AI rather than building AI?
Ed: Yeah, I'm going to appeal to what I usually tell people, which is education is the ultimate equalizer. So, nobody can pull the wool over your eyes if you learn about the thing that they're trying to pull your wool over your eyes about. So anybody who wants to get involved in AI will never discourage from getting involved in AI.
At what level and what depth and where is going to be independent. It's going to be, or I'm sorry, dependent on what it is that you want to do. So get involved. This technology is going to be ubiquitous. Right. So your question around, do we, do we need a data scientist? I will tell you that in the world of statistics, an automated statistician has been a dream for the better part of, I think, a hundred years or so at this point.
So, like, we've been trying to automate the process of statistics, just based statistics for a very long [00:44:00] time. And I think that we will continue doing that. Right. And when you look at like alpha geometry that came out of DeepMind, or you look at alpha, what's the name of it? I can't remember. There was one that just won the math Olympiad challenge, or at least was able to place within the math Olympiad challenge.
We're creating, it's interesting because we're creating idiot savants. All right, like these models are really good at solving these mathematical challenges that humans are able to, you know, participate in with like math Olympiad, but then you ask it what number is bigger 2. and it'll give you the wrong answer.
Right, so back to should people get involved in AI? Yes, it's going to become a technology that is not going to Pando's box has been opened. And we're not closing it anytime soon. I do believe that write once, run once code is coming. So if you were to sit down and need to accomplish a task, and that task requires you to write some code, you can use one of the code assistants that you have, and if [00:45:00] you execute the code and the job is done, you don't need the code anymore.
Right. So then that changes the way you think about things like test suites and bugs. Are there going to be, is it necessary to have such a robust set of test suites for these particular sets of code? As long as a human sits down and verifies that the output of the model accomplished the task. If it did move on, you don't need the code anymore.
We'll do it again next time. Clearly the model embodies the ability to perform this task. So we can assume that in the future, the model will perform that task at least as reasonably well as it has right now. This is a big assumption to make clearly. So I will never discourage people from getting involved in AI.
It's going in any way, whether it's the user side or the development side. It's not going away. I know Andrej Karpathy called it software 2. 0. I truly believe that to be true as well. And I also believe that large language models are becoming a new unit of computation. And what I mean by that is It's much like you have a CPU complex inside of a computer.
You send bits in and bits come out [00:46:00] and you expect to do something with those bits. We're doing the same thing with large language models, right? So I think that they are going to become a fundamental unit. Or some derivative of large language models in the near future, going to become some sort of fundamental unit of computation.
In that case, everybody should try to be as familiar with that as possible.
Bill: And it takes us to, I don't remember who coined the stat, but it was like 70 percent of the jobs that are going to be available in 30 or 40 years haven't been invented yet.
Ed: Right.
Bill: And so, like, what are you even preparing for? Don't know.
But it's going to be something that at least touches on AI in some fashion because. Yep. That's, that's the next steam engine, which is pretty amazing.
Ed: It is. It's interesting to watch unfold. It's, it's, I tell people all the time, it's a wild time to be alive right now.
Bill: Yeah, for sure. For sure. It's really moving fast and changing and it's interesting to talk with someone who's sitting right on the forefront of it like you are, which is.
It's just super cool. [00:47:00] Thank you so much for joining us today, for sharing your perspective. And this was just a fun conversation. I learned a lot. I took a couple notes. I've got some things to follow up on. Mostly I need to read philosophy apparently.
Ed: That's just a bias that I have. I like to think about why a lot, even in, you know, science eventually you, you ask why enough and there's no answer. So you, you have this beautiful dance between science and philosophy. That's almost necessary. And then actually the last thing I'll tell, I'll tell you, Bill, about the kind of continued churn of this process is David Deutsch.
He's a philosopher, I think, university of Cambridge. I can't remember somewhere in, in, in Europe. And he gave this great example around how things like the technology that we have today in terms of AI are missing something fundamental. And he uses the neutrino problem that existed during the 60s and 70s, I believe, as a motivating example.[00:48:00]
So at one point in time, the, I don't understand the physics enough, so don't quote me on it, but you can go look this up. I did the research. It's a phenomenon that existed. The, the model that we had to describe the sun and the radiation that it's throwing off. The mathematical model that we had basically stated that the sun shouldn't exist.
That would be a problem for us. But we're looking at it in the sky, like, no, no, no, it's there. But like, the model that we have describes it as such like it shouldn't be there. Right? And it's because the neutrinos that we had discovered that the sun throws off had a certain property that they would decay as they headed toward Earth.
But we didn't have this decay factor in our model. So, pause. If we were to take a large language model back in time and take it to the 60s and 70s and we were to train on all of scientific discovery up until that date, and then we were to ask that model what the problem is, it wouldn't be able to answer the question.[00:49:00]
And the reason why is because the answer to that question was a concept that a human had to invent, which is that decay rate, right? So this whole idea of being able to create a system that is autonomous and intelligent enough is. It falls apart in the face of what machine learning is today, which is learning from historical data, because sometimes you can't use historical data to make the decision you need to make moving forward.
These things are not able to create in the way that a human is to create, at least in that abstract kind of way. And I really liked that example from, from again, David Deutsch, anybody who wants to go read up on some of the work that he's done. To your point about philosophy, it's, I think, because we are trying to recreate ourselves, we are naturally beholden to this, like, inclination to think about the philosophical aspects and perspectives of what it means to create ourselves.
Bill: Existential science. That's right,
Ed: man. Philosophy of science.
Bill: Cool. And thank you so much. [00:50:00] This has been a fun conversation.
Ed: Yeah. Thank you, Bill. Thank you to everybody who listens. If you ever want to get in touch with me, I'm on LinkedIn and Twitter and I'll provide all that information to the folks here on the podcast.
Thank you again to the crew for having me and I look forward to any type of future endeavors that we have together.
Bill: Good to know. Thank you.
Ad read: Capitalize on your edge to generate more value from your data. Learn more at dell. com slash edge.