🎧 #067 - Trevor Pering on 3 ways Google is enabling Enterprise IoT

"When the iPhone came in, the web was already there. With building systems, it's the other way around. You had a lot of technology in buildings, then we bolted the web on afterwards.

That's a different direction, so it's trying to correct that and bring more web concepts in."
—Trevor Pering

Welcome to Nexus, a newsletter and podcast for smart people applying smart building technology—hosted by James Dice. If you’re new to Nexus, you might want to start here.

The Nexus podcast (Apple | Spotify | YouTube | Other apps) is our chance to explore and learn with the brightest in our industry—together. The project is directly funded by listeners like you who have joined the Nexus Pro membership community.

You can join Nexus Pro to get a weekly-ish deep dive, access to the Nexus Vendor Landscape, and invites to exclusive events with a community of smart buildings nerds.

Episode 67 is a conversation with Trevor Pering, IoT expert at Google. This is a great third edition to our ongoing Google series of podcasts.

Summary

We talked about three types of IoT and about how enterprise IoT is different and the challenges related to enterprise IoT that Trevor focuses on, such as managing devices, automating tasks, cyber security, and scalability.

Then we dove deep into three of Trevor's main open source initiatives, what they're designed to do, and how they can help transform the buildings industry.

Without further ado, please enjoy the Nexus Podcast with Trevor Pering.

Mentions and Links

1. Nexus Podcast 29 with Trevor Sodorff and Keith Berkoben (5:31)
2. Nexus Podcast 56 with Sabine Lam (5:41)

You can find Trevor Pering on LinkedIn.

Enjoy!

Highlights

• Trevor's focus at Google and differentiating the 3 types of IoT (5:56)
• Deep dive on DAQ - device automated qualification (13:26)
• Deep dive on MUD and the requirements for IoT networking (24:42)
• UDMI (Universal Device Management Interface) standard with device<>cloud communication (34:58)
• Trevor's take on where we are on the disruption curve and how he chooses partners based on whether their business model will hold back innovation (46:54)
• Trevor sums up all three projects (55:03)

Music credit: Dream Big by Audiobinger—licensed under an Attribution-NonCommercial-ShareAlike License.

Full transcript

Note: transcript was created using an imperfect machine learning tool and lightly edited by a human (so you can get the gist). Please forgive errors!

[00:00:03] James Dice: hello friends, welcome to the nexus podcast. I'm your host James dice each week. I fire questions that the leaders of the smart buildings industry to try to figure out where we're headed and how we can get there faster without all the marketing fluff. I'm pushing my learning to the limit. And I'm so glad to have you here following along.

[00:00:31] James Dice: This episode is a conversation with Trevor Pering, IOT expert at Google. This is a great third edition to our ongoing Google series of podcasts.

We talked about three types of IOT and talked about how enterprise IOT is different and the challenges related to enterprise IOT that Trevor focuses on, such as managing devices, automating tasks, cyber security, and scalability.

Then we dove deep into three of Trevor's main open source initiatives, what they're designed to do, and how they can help transform the buildings industry.

Without further ado, please enjoy the Nexus Podcast with Trevor Pering. Hello, Trevor. Welcome to the nexus podcast. Can you introduce yourself please?

[00:01:09] Trevor Pering: Yeah. My name's Trevor pairing and I'm a software engineer that works on Google looking at a number of related technologies for building IOT systems.

[00:01:19] James Dice: Great. Great. And can you tell me a little bit more about your background before Google?

[00:01:24] Trevor Pering: yeah, you know, I'm career life, or I know we got a life for it, but long-term person, which is a bit rare in some industries nowadays. So I went to college at UC Berkeley and I was there for 10 years, which is quite a long time. And that was, you know, my, both my undergraduate and graduate work.

So I wasn't stuck. I was just, you know, I liked the place and had a lot of fun there and I was there for 10 years. And after that, I worked at Intel for 10, 11 years as well. So I was quite a long time working at Intel and up research organization, looking at some interesting, you know, always pushing the edge of new technologies and stuff like that.

And then for the last 10 years or so, I've been at Google just past my 10th year anniversary on August 1st. So, so yeah, and it's a different perspective on some of these systems. My background is somewhat a mix of electrical engineering and computer science by technically a degree is in electrical engineering, but really I'm computer scientists at heart.

And it's, you know, that it is a different perspective in terms of that, but I always have this root and foundation in physical things and. When I look back on my career, a lot of it was just falling through whatever it was in front of me. You see something interesting, you go for it. That's why I ended up Berkeley for so long.

As I found a professor that I really liked and started working with him on things. I was doing a lot of fun extracurricular activities. And so it was just this natural thing to fall through looking back on my career. The only real constant I could find is that there's a definite relationship of virtual and physical world together.

All right. And one thing, for example, opposite, that's something I hate in terms of technology. And it's great for other people and it's nothing against it itself, but things like VR, I just, they really don't resonate with me because it's the wrong direction. You know, you're, you're trying to create this virtual world and block out everything physical and all of that.

Totally. Well, I go the other way, and this is why, you know, for all those wanderings and stuff like that, On this project, working with building systems is it's almost, in some ways the ultimate merging of these things, you have some of the biggest physical things you could possibly think of and trying to make them digital and virtual in some ways.

Brilliant. I love that.

[00:03:50] James Dice: Uh, We'll get into all that in just a minute, but speaking of extracurriculars, I read on your LinkedIn profile band band and more band. Can you, can you just talk about the instruments you play?

[00:04:02] Trevor Pering: Well, instrument. Yeah, so I play trombone. It's, it's been a while, but you know, I played it and it was a great experience.

So, so this was primarily the Cal university of California marching band go barracks. And it was an absolutely great experience. Some things in there that can, I don't know how they'd ever be reproduced in life. And you know, this, this is along the lines of maybe give you a little hint about my personality sometimes.

The trombone can be very loud. It is, I think physically one of the loudest instruments, just given the residents that it can, it can produce. And there was a moment, one time when I was on the field of the marching band at halftime of some show and just playing crazy loud. And I know, and I've heard recordings and stuff that you can, and this isn't always a good thing.

You can hear me. And, and you know, this is out of a field of one hundred forty, a hundred fifty people, and it's that kind of experience, right? You're, you're there in a physical world with people, all the excitement. I don't know. That's partly why I did it for so long and so many different forms. I think I was actually in the band for, I forget what it was six or seven years.

Which is longer than most people are an undergrad. Right. And I was there for grad school, so, but it

[00:05:20] James Dice: was, it was a great

[00:05:21] Trevor Pering: experience. Very cool.

[00:05:24] James Dice: So I want to like set the context for you. You mentioned the project, but you set the context for your work at Google. So, this is kind of a follow on to the episodes with Keith and Trevor from last December, which we'll link to in the show notes.

I believe that was episode 29. And then we did one with Sabine. I believe that was in June. Don't remember the number of that, but I'll, I'll link to that in the show notes for everyone. So I want to kind of follow on you came highly recommended by Sabine to kind of dive into some of the stuff that you're working on.

Can you talk about like in general, what's your, what's your main focus in supporting Google's buildings?

[00:06:00] Trevor Pering: So for me, I am really focused on a bunch of the infrastructure pieces that go into connecting all of these devices securely and manageably to the cloud. Right. Um, Keith, and what they do, I describe, and I'll use the term big data to talk about what they really think about what they care about.

And this is where you get into the data processing things, all the intelligence and all of that, right. I'm focused down on the look, I have a device here. I need to get data from that device into the cloud. And what does that mean and how that plays out? Totally. When I think about these systems and I'm loosely termed as enterprise IOT, right?

So there's a lot of stuff that's actually outside of the building space, but it really comes down to enterprise IOT. It distinguishes itself in my mind from industrial IOT. And consumer IOT. So there's three different flavors that are really feel industrial. IOT is probably the most straightforward on in which is just quality characterized by a lot of the same thing.

So you can have a million sensors on every single don't quote me on the numbers, but every single oil rig out there, every single one of those things is the same thing. Lots of data. It's, it's massive in scale, hugely impactful, but it is very regular and consistent. Right? Then you have consumer IOT, which is much smaller.

This is your, your apple home, your Siri assistant, Google assistant, all sorts of other technologies that go into the home. You know, the focus there is around personalization and this is providing a really coordinated, great experience that really fits what one particular person wants or a family. Right.

Then there's a unit there, but it's very. Focus perspective on that kind of space that we work in is enterprise IOT, where it's just like, okay, I have a whole bunch of stuff from, you know, easily 10 manufacturers, building devices in one building. And you have whole bunch of employees, you have crazy things going on all the time, constant churn vendors coming in, changing things, fixing things, all of this stuff.

And it becomes this massive ball of chaos that you have to work through and reason about. And that's, you know, very endemic to what enterprise means in a lot of ways. And this is true about side of IOT as well, but that's really the focus of a lot of it. So from my perspective that I I'd take it down to essentially security manageability are, are some of the core, the core pieces that I really get at and really focus on and really just trying to make it make it possible to run these systems at scale, essentially.

[00:08:41] James Dice: Cool. And are there some patterns from those other types of IOT that could help us sort of understand the, like what you're trying to enable? Or is it totally, totally

[00:08:52] Trevor Pering: different? No, no, there's definitely patterns and that's a large, you know, side of what we do. And, you know, so I talked about industrial and part of this is, you know, understanding industrial thing that the data scale is massive there.

When you compare that to enterprise IOT and what we're doing with buildings, the amount of data that is produced and consumed by industrial things is, is much higher, which means a lot of the infrastructure and things aren't in place for dealing with a lot of that data. Okay. On the consumer side, you have a lot of things that go on that really come down to trying to make the system secure for, for what it is that someone wants to do.

And. And I think it's interesting to go back and I know a couple of the previous discussions or talks about the iPhone and how that really changed some of those consumer landscapes. One thing about the iPhone to remember is that the first launch of the iPhone and the first version that came out had no app store, right.

It was email maps, web browser, and a few other things. And if you talk to people now, what they find most amazing about it, right? Cause a little ecosystem of apps that it enabled and that just wasn't there at the beginning. And that's because what it really was is a really solid foundation for this is what a really good product needs to be based on.

Okay. Right. Both technical design and mechanical engineering and all of that. And I feel we're, we're need to apply some of that same reasoning to what we're doing with these buildings now, which is provide that really solid foundation for something. And then things like the app store can come later. And I know um, one of the previous podcasts too, I was listening to John Clark was talking about this as well, which is people want the shiny thing, but that's, that's a want things like security or a need.

And, you know, you can think of it like a safety belt, no one wants a safety belt. You know, that's not why you're sitting in a car is to have a safety belt, but you need it. And it, it, it becomes something that you have to have. And that's how I view a lot of the work that I do in terms of the security foundation for what goes into these buildings.

[00:11:07] James Dice: Very cool. I love that analogy, the safety belt. Um, Yeah, and that does, if, if no one has, if people listening to this have not listened to John, I th I believe his analogy was you can't build the app store before you build the iPhone like that. That's not the order in which it happened. So I love that, that quote as well.

Shout out to John. I know he's a regular listener, so he's going to love this. So it sounds like generally, if I can repeat this back to you, generally, we have all these different devices that are being put in the buildings, right? HVAC lighting, access control meters Peloton machines you know, down the line, right.

And those are being put into the buildings and you're trying to enable the ability to connect them to the network, manage them and make sure they're secure and keep them updated over time. In general what you're trying

[00:11:59] Trevor Pering: to accomplish. Yes.

[00:12:01] James Dice: yeah, so let's dive in to the weeds if you want to.

[00:12:04] Trevor Pering: Yeah, yeah, yeah. And you know, one way I think about this, some is to, again, so Google of course, right, is a massive web company and our roots come from the web and what that means. And so there's a lot of analogies that I like to pull forward from web systems. And you can go and look at what the web does around X and what the web does around why, and bring that forward to what we need to do with IOT systems to get us to the same place.

And I, I know some other people in previous podcasts has mentioned this too, that the technology that we think of as IOT and digital buildings is decades behind. Where we are with consumer systems, right. Huge gap. And I like to explain that with the concept that, you know, really, when you look at the consumer world, we started with the web and, you know, so really amazing back end things and kind of brought it to the physical world.

When the iPhone came in, I was like, look, the web was there. Now we're adding in this phone with the building systems. It's the other way around, right. Buildings were there first. Right? Right. You had buildings, you had technology and buildings. You had a lot of technology and buildings, then we're bolting on the web afterwards.

Totally. Right. Yeah. That's a different direction. And so it's trying to correct that a little bit and like, okay, now I need to bring some more of that web stuff, John, into the building and make it all right. One of the first things then that we do and need to do with this is what we call qualification.

And specifically a tool that had been working on Kodak, which is an open source project that is about it's. It stands for device automated qualification. And the idea here is this is a thing, and I know Sabine mentioned this previous, sorry, this is the thing you can plug anything into. And it tests that device against a set of criteria that we have and evaluates how, how good it is.

Right. And that's all it does is just testing the device. And from, from the web perspective, you know, if you look at HTML specification, what is your web browser really support, right? It's kind of the same thing. You can go to these pages and it'll evaluate your web browser and give you score. And nobody's perfect, right?

Everyone is doing little bits and pieces of the entire thing. It's a massive spec, right? And we're kind of in the same space here, which is, look, you have this device and you don't want to bring this device into a. You need to do X, Y, and Z at a bare minimum. And you need to, you know, we want you to do these other things, to make it more functional.

And then eventually you want to get to these advanced features such as, you know, the firmware updates or whatever going on. And we need a mechanism to test that, to get that out there into the world, to clearly specify to everybody what it is that we want them to do. Right. And that's where we get to this qualification tool for that.

Okay. And how

[00:14:51] James Dice: many of these checks are there just to give people

[00:14:54] Trevor Pering: context

lost count there? I think there are, you know, the overall qualification process and everything that gets checked is, is hundreds of things. Right. And, you know, the problem is only some of them can be automated. And so, yeah, there's a handful of them. I think it's around 40. I want to say various tests and they go all the way from things like, do you do MTP program?

Do you do DHCP properly and you'd be surprised, you know, DHCP is a 20 year old standard at this point. There were a lot of devices that did not do it right. And, you know, they would fail on our corporate infrastructure because of that. So we have some tests like that all the way up through default password, checking through pass port scanning, open ports, checking, proper TLS certificates.

You know, anything we could think of the automate get, gets put into this to make it a better product. And, and

[00:15:52] James Dice: these things are what would pop up when, you know, five years from now, someone said, or maybe even a month from now after the building gets set up, someone says this device isn't communicating, or this device you know, dropped off the network.

These are the types of things that would cause errors in

[00:16:10] Trevor Pering: some way. Am I right? Yes. And there's different stages of this. And there's four main buckets that we put things into for, for that. One of them is basic compliance, which is like, look, if you don't do this thing, the system just will not work. You know, it'll flat out break.

And that's a lot of what you're talking about. There there's another layer, which is essentially security compliance. It will work, but our security groups will not like it. It is open to compromise and all sorts of things. So that's a different bucket of things that we evaluate. There's another bucket of things that is application specific protocols.

And so this, for example, to very much be like, is it supporting various bits of the backnet protocol correctly, right. And those are important when you want to get the whole system working together. The fourth, fourth bucket, which is the, you know, the brave new world of crazy new stuff, which we'll talk about a little bit later.

And some of the other pieces we're working on, which is about how does this device connect to the cloud, right. And what does that mean? What data format do we want to do? How are things structured? All of that stuff is totally wild west for people. And there it's not so much, you know, a security and the networking stuff.

You can say, if someone's doing it right or wrong and more of an objective sense, this is really look like, you know, if you just let everybody do their own thing, you just end up with a big mish of all sorts of stuff. Absolutely. So this is again using it to help tame this field of what everyone's doing about things.

Yeah.

[00:17:38] James Dice: And there's a ton of people listening right now that are, that are working on projects. Like I am where you just have data issues all the time and they're going, man, if I would've had this before this project started, that would've been a great thing. So how does this get set up? So Sabine mentioned, you guys have something like 700 buildings.

Some of them are existing. Some of them are new. Some of them, your tenants in how does this get set up and used on these different categories of Google bills?

[00:18:04] Trevor Pering: Yeah. So, so there's a whole pipeline of how this tool is intended to be used and very specifically, and this is why it is an open source tool and it's completely free for people to use.

Anybody can use it, right? So the whole pipeline actually starts with manufacturers using the device or using the system to test their own device. And that just gives them a better sense of where they're at. Then we have a stage, which is the qualification stage, right. Which is a centralized lab of some sort where people send devices and we test them against these standards to make sure that, you know, we think the manufacturers did the right thing, right.

That's all pre building. And pre-project right. What that means though now is we have a pool of devices that are known to work, right. Then we go to the next stage, which is the project stage. And this is where it starts to get extra complicated with all this right. A project executive or, you know, whoever's doing the project has to decide on the executive, but you know, someone who's working on that project specifically has to pick, look, I'm going to use this model of AGU.

Right. They have to figure that out. And then they have to go off and get a thousand of these things and somehow assemble them all into one cohesive system. Right. Which is the building. So the next evolution of this from the, you know, use this tool in the lab is to start to use this tool on more complete networks, right?

So here's a whole network of devices and how these things get put together. And that's the path towards what you're asking about, right. In terms of how does this manifest itself in this massive fleet of buildings or something like that. Right. Which is we have to look at additional kinds of automation so that we're not just automating the test between the test harness and the domain.

But we're automating the complete process of testing a thousand of these devices as installed on a, on a, on a network. Right.

[00:19:56] James Dice: Totally. Got it. And I can imagine when you're kind of pushing these requirements out to the construction and operating supply chain do you have any pushback? Do you have any like real stories from implementing this and well,

[00:20:13] Trevor Pering: yeah.

Yeah. It's a really interesting dynamic with that and it comes down to, and there's three buckets of responses that we get, right? One of the buckets is basically we, this is not for us, you know, we're just not going to play along for whatever reason. And you know, this tends to be. Some of the smaller companies and manufacturers just simply don't have the resources to push on things in a speculative way, which is some of what we're doing with some of this, right?

Then you get a tier of manufacturers that are essentially about innovation. And these are the ones that recognize that the current status quo is not long-term viable. And they're thinking about what is the business and market going forward and where, where are things going to be and where do they need to go?

How do we solve these problems that everybody recognizes, then there's the third bucket. And then those are the people that we tend to work with the most because the third bucket, which is going to be the big players, right? And these are the people that essentially have competing standards or systems.

And we end up in these, these back and forth conversations where we 100% agree on. The problem that needs to be solved there there's very little confusion around, do we want over the air form or updates? Everyone will say, yes, the problem comes with, oh, well, why don't you just use my system to do that versus no, but there's this other way.

And then there's 20 ways of doing it. And the bigger the company is, the more they have typically invested in some other solution. And that's where it gets really tricky too. So we do work with them. It's just a very different conversation than that, that middle tier people that throw, we find the most engagement.

And from your

[00:21:55] James Dice: perspective was with such scale in the buildings you're working in, you can't afford to have all these different ways of doing things that kind of a philosophy.

[00:22:05] Trevor Pering: Yes. And a lot of it is about finding the. Least common, multiple of all these devices. Right. And it's basically look at the end of the day, the AHU and the FCU and the chiller and the Peloton machine and the coffee machine and the light switch and the light actually all the same.

They, they, they, they have a common set of properties and what you want to do is make those things all uniform and then find the right ways to allow the differentiation of them, but not make that kind of part of the standard. Right. And this is the problem we ended up with now a little bit when you're like, well, this set of devices uses back net.

This one over here uses Dolly. Then there's some other protocol for this other sub system. And it's all these vertical silos of things. It just drives us crazy. Right. And, and how do you come up with that? Right. Like, look, here's the baseline thing that everything should talk and we'll go from there.

cool.

[00:23:01] James Dice: So, if someone wants to use this, it sounds like they can, it's readily available for them to use you know, on non Google projects to sort of borrow from the innovation that you guys are, are

[00:23:13] Trevor Pering: doing. Yeah. Yeah. And, you know, we, we have actually worked with some of our consultants when I worked with them, but some of our consultants that we work with are doing that as well.

You know, they turn around and they use some of these systems and tools to work with some of the other projects that we're working on. Right. And we have worked also this, like I mentioned with manufacturers directly to get them to use the tool because it doesn't really matter if their product at the end of the day is used on a, on a Google building or some other building.

Right. Everyone wants these things to be secure and more managed more. And so it's really trying to get that ecosystem of these things going. I recognized that, you know, Google is big in the real estate thing. Right. But we're not so big that it makes sense to have anything proprietary in this space.

Right. It just, these things need to be industry wide. Right.

[00:24:05] James Dice: Right. So then at some point maybe you can stop needing to check.

[00:24:11] Trevor Pering: Yes. Yes.

[00:24:12] James Dice: Okay. All right. So I'm thinking about a given project. It's got all these devices now these devices have been sort of certified using Dak. And so then, then what, what's the, what's

[00:24:24] Trevor Pering: the next frontier?

Yeah, no, I don't want to get pedantic here, but certified as something different and you have to be careful. A lot of places certified is a legal definition of something, right. That's why we call it qualification and it's a tricky, subtle thing, but. The next stage. And what I think about is essentially assembly, which is how do you take all of these individual devices and assemble them together on a shared, managed corporate network, right?

This leads to some of the other systems that we talk about in terms of what it means to manage network security. And there's another standard out there called mud. This is an IETS standard manufacturer usage description, right? Which starts to get at this interconnectivity between all these devices. And it's basically saying, look, this device over here talks back net, and this other device over here also talks back that, and it lets the system then understand and reason about what that means.

And so it's not just this wild west of everything can talk everything all the time to anybody it's really about getting much more specific around what can and cannot talk to each other on the network. And that's really important when you start bringing all this stuff together. Interesting.

[00:25:38] James Dice: Okay. And.

Just because I'm not from this world. I think I'm a little confused around the differences between this and sort of the interoperability or ontology efforts, right. Where I'm saying, you know, I am a air handling unit and these are my points. Is it where's the overlap with that

[00:25:56] Trevor Pering: effort. So it really comes down to what layer of the stack that you're thinking about.

Okay. And this is, you know, so for example, I'm talking about, can this backnet device talk to this other backnet device? Okay. Right. And that's a fairly low layer in the stack, right? Well, you're talking about with intelligi, right? It's like, well, the data point that this device, you know, reports, how does it relate to this data point that this other device reports that's a semantic layer.

One is transport layer mechanics. The other one is semantics. And the analogy that I use with this is like, if you look at web technologies, So you have HTML or HTTP, some of those pretty low-level things that are pervasive everywhere. Right? That's great. But it says nothing about the content of what you're sending around and the ontology and all those pieces are really the semantic content, not the transport layer pieces that are lower in the system.

Got it. Okay.

[00:26:58] James Dice: So what are some of the ways that the, as you call it mud one of the, some of the ways that mud sort of changes the status quo with how a building would be built, if it weren't

[00:27:07] Trevor Pering: for it. So it's interesting. Cause so what it basically means is that every single point to point communication in a building needs to be specified.

Right. So if I have a hundred devices that are all talking back net, right? It is rarely the case that all a hundred devices need to be able to talk to every one of the other a hundred devices, right? That's not how it wants you to have devices that talk to the head end. That's a different matter. And what typically happens in building deployments, everybody, you get an open network, everything goes on that network.

Backnet has protocols done for discovery and broadcast and all of the seven ends up being a big mess. It works, but it's a big mess of things which is bad for security, right? So it has to happen then is to say, look, this device has to talk directly to this device. And it's, you know, basically aligned between them instead of this open number concept.

And that's something that when you talk about the networking side of things is, is quite a struggle for a lot of installers out there because it's just so different than what is done today. Totally. What do you think is though when you actually get down to, and you sit down with them and you show them, oh, this is what I want.

They're like, oh, I get it. It's a wiring diagram. And if you go back to the days when things were wired with, you know, RS 4 85 for backnet, so not IP, but back it was the exact same thing. You had boxes and had literally had wires between them. Right. And so conceptually, they get it and it's really not a hard sell conceptually.

It's just that they got used to this kind of fluffy network layer in between. And that's, that's essentially the whole, we're trying to plug with this. Got it. Okay.

[00:28:48] James Dice: And that is, and again, not from this, this area of the industry. So I'm always asking sort of beginner level questions here. This is regardless of the physical network that's installed in the building, you know, you're, you're talking across different sub networks within the building, is that right?

[00:29:06] Trevor Pering: Well, I, you know, that's right. It does get tricky. And a lot of the details, it's basically it is the same physical network, right. And this is where we run the ethernet cables to everything and do everything over IP. So it is a uniform layer at that level. Right. And that's done because it makes the construction side of things so much more regular and predictable.

Right. You're doing the same thing with all cables. It's all Ethan cables. Right. All this differentiation then comes at a layer a little bit above that. Right. Which is like, wow, this is backnet over IP talking. This one is some other protocol over here talking. And, you know, that's where it gets down to these differentiations of things that like, okay, fine.

You have everything in this one big room. I can talk to each other virtual room what actually needs to happen with that. Okay. And this is all again, local in the building, right. So we haven't even gotten to the cloud smart part of anything at this point. Yeah.

[00:29:57] James Dice: And I can imagine, you know, some of the members of the supply chain are used to,

Having their own network, they set up, that's sort of separate from, you know, the IOT network.

Right. And so you guys have basically said, there's only one network as well.

[00:30:12] Trevor Pering: That is the whole headache that I just personally avoid. You know, I'm a technologist and I develop tools and systems and stuff like that. I hear the stories from the trenches of what that means and how hard it is. And I was like, oh boy, you know that, fortunately that is not my job.

But I, I feel for that thing, is it, it is a change in some of these processes that are very well understood, but just baked in and a hard, you know, Absolutely.

[00:30:35] James Dice: And so, so is, is mud the same as DAC where someone can implement this style, the standard on their project outside of Google or what's, what's sort of the state of deploying

[00:30:47] Trevor Pering: and sharing.

Yeah. So, so mud is a open standard and that's an IDTF standard, right? So very, very well-regulated one. The problem is it is very open-ended and it's one of those things it's like, well, okay. Just because you can do that, isn't what we want to do. So there's a question of policy and then there's also the trickier one that we're still working on and how it relates back to Dak actually is one of, of verification validation, right?

So mud gets implemented as a Jason file that describes what this device does or can do, right. I can run a Jason file. It doesn't mean it's going to be right. How does someone know that the Jason file that they produced for a device that the MSI then has to assemble together and some bigger system is doing the right thing.

Got it. That's where we get back to DAC then as this qualification tool and testing automation. So one of the threads for example, is what kind of mud file qualification support do we have as part of Dak? Yeah. And it's not there yet, but that's one of the pieces we're working on for very specifically that reason, right?

How do we enable everyone to go take this open standard? Lots of examples of what files could look like, but actually get it to the form that it needs to be, to be successful on one of these construction projects.

[00:32:06] James Dice: Got it. And how do you, and my mind's going straight to commissioning here, right? Someone checking, I don't know how familiar you are with the mechanical commissioning, but my mind's going towards, like, you guys have internal people that are focused on doing this qualification on each of your projects, how does the industry, and this, you don't have to have the answer to this.

There could pontificate, but how does the industry is sort of play that commissioning role using these tools to basically make, you know, leverage these tools that you guys are creating?

[00:32:36] Trevor Pering: Yeah, that's a, that's a really great question. I'm very familiar with that. And that is one of the areas that I am actively thinking about.

I'll pontificate all day long about it. If you want. Basically, what I see right is what happens is you get into this environment where you have three or four different layers of the system and each one of those layers has someone who's kind of responsible for that system or that layer. And the issue is that when something goes wrong, it's like, well, does it.

Why I don't know, point fingers at somebody else because no one really knows what's going on. So a lot of what I've been thinking about is how do you develop tools and systems that essentially can up level all of this stuff into one place where you get more of a centralized view of what's going on.

Right? So for example, I think the ideal use case, or, you know, end use of this stuff would be essentially that like someone who is installing a device or trying to diagnose a device can go up there with their phone, scan, that device, a QR code on there, or something, this links into the cloud somewhere, and then just shows them here's the diagnostic output for that device, right?

It's on the net. Um, But it's still not functioning correctly. Right. And so it helps permission the problem to know who do you go talk to to fix it. Right. Cause if it's a network problem, then you have to go talk to the network person. If it's a backnet configuration problem, that's somebody else you have to talk to.

And it's really about how do you treat AAJ these, these problems into the right bucket to figure out who to go talk to. Absolutely.

[00:34:06] James Dice: Again, I think a lot of people are probably hearing you talk right now and going, yes. I know

[00:34:12] Trevor Pering: exactly that that's a very common response I get, but you know, the tricky part is actually doing something about that.

It's, it's a lot of moving parts and a lot of dishes that have to get put. Yeah, totally.

[00:34:23] James Dice: Hey guys, just another quick note from our sponsor nexus labs. And then we'll get back to the show. This episode is brought to you by nexus foundations, our introductory course on the smart buildings industry. If you're new to the industry, this course is for you. If you're an industry vet, but want to understand how technology is changing things.

This course is also for you. The alumni are raving about the content, which they say pulls it all together, and they also love getting to meet the other students on the weekly zoom calls and in the private chat room, you can find out more about the course@courses.nexus lab. Start online. All right, back to the interview

[00:34:58] Trevor Pering: You know, the, the next logical thing, Was talking about is in order to do this and this application I just described. Right. You have to get data up into the cloud in some way, right.

Because that's where you can centralize it. And that's where it, it comes all, comes together to form this more holistic view of what's going on with all of this stuff in the building. Right. It doesn't work to have an isolated anymore, and it doesn't work to have people always poking with things in the building.

Right. So there's another layer then, which you get out, which is well fine, how to all these devices and all these systems communicate to the cloud. Right? How do we get to that level of. And this is the foundation of what we talk about a smart ready building, right? Yeah. I'm not saying it's smart because this is not about machine learning or about FTD.

And some of those applications that people have talked about previously, right? This is about, look, these devices can get data into the cloud and we can tell that it's getting the right data to the cloud or they're configured correctly. Right? So it's a foundational layer under this and leading into this, then we have a, another open standard that we've been developing called EDMI, which is universal device management interface.

Okay. And what this is right, is just a set of conventions to say, Hey device, if you want to get your data from where you are up into the cloud, please do it this way. Right. Here's a Jason format, some schema around it. You know, this doesn't quite get to the intelligent layer you were talking about of, oh, what is your point actually mean?

But it's, it's a layer below that. And it's just a standard way to get that data up into the cloud.

[00:36:35] James Dice: At what point do you think the device manager, and I've been asking this on several different podcasts.

When will device manufacturers have the ontology baked in so they can communicate it or like, do you always think that's going to have to be added in the cloud?

[00:36:50] Trevor Pering: Well, you are asking the wrong person. Okay. Yeah, that's a, that's a Keith, you know, Sabine's one of those previous other people question. Okay.

So.

[00:37:00] James Dice: Yeah, I, I keep searching for that. Maybe I just need to start lining up manufacturers and say what point at what point, where you start to describe what you're doing uh, in an open way. All right, so, so what are the keys to sort of this piece that are, you know, you guys are using this UDM to sort of improve upon what here what you're seeing out of device manufacturers, right?

Or what are the keys to

[00:37:25] Trevor Pering: this? Yeah, so the keys are one is that is a very basic layer of requirements around just how to do things in a secure, direct way. Right. And this is basically like you have to use this kind of encryption. To this validated endpoint and then use MQTT on top of that and then encode it.

And Jason, there's a very low layer of this that is just like pure basics. And there's no rocket science here. You can all, you know, anyone can go look on the web for all the completely open standards. It's just getting them actually figuring out exactly what needs to happen. Okay. But once you say, please do this, then that, that ball starts rolling and people can do it.

Right. We've had lots of manufacturers that once they decided, oh, we're going, gonna do this. They're like, oh yeah, we did it. And it was a lot easier than we thought. Right. So a lot of it's just getting over that, that FID layer of doubt and like, ah, who knows what's going on right now? Yeah. So that's one piece.

The other one. Yeah. It starts to get a lot trickier. Right. Just coming up with just agreed upon standards for how to do something. And a great example of this is what we call call right back. Right. Which is, look, we have a thing in the club. The ones that changed the set point of that thing down in the building to 20 degrees Celsius or whatever.

Right. Okay. That's a pretty simple idea, but it turns out there's a bazillion little micro decisions that have to be made to get that done. And it just, and part of it is you got these two worlds colliding where there's like, Hey, these people in the business and the IOT building business have tons of experience with this.

And then there's me from the cloud going, Hey, I think this would be easy. Let's do this. And we come together and we have regular open kind of working group meetings where we talk about this stuff. And just this morning we were talking, I was like, Hey, how are we gonna do this? How do we encode this? How do we test it?

And just coming up with, here's the thing we, we kind of agree upon to do this very simple, simple idea, right? Yeah. So that's another layer of key thing around what's necessary to make this work. Right. Which is really simple, clear specifications. One problem. We typically run into with this we're be like, Hey, we want to do this really simple thing.

And someone says, oh, that's great. We have this massive library framework that does that. And you're, you're bringing in this huge dependency on this big library and all this complexity just to do the one simple thing. Yeah. And so we're, we're doing a different dance with that. We're starting simple. And then maybe it'll take us longer to get to the big thing, but we're, we're trying to get the basics.

Right. Got

[00:40:03] James Dice: it. And how does help with the right back? Right. Cause when I'm thinking of all the obstacles to writing back, I'm thinking of. Each layer having its own obstacle itself. And then when you get down to the system all the different modes of control, sequences and ways in which set points are stored and priority levels for set points and all that.

So what does UDI help with in that mess?

[00:40:28] Trevor Pering: Yeah. So there's two basic things that, that it really helps with right. One is that it really tries to simplify what that exchange is, right. It says, look, I want to write this point. I want, at this point to be 20 figured out, you know, I don't care how you do it. I don't know what you need to do.

And this is me pretending to be the cloud, talking to the device on prem. Right. I don't care. Just make this point 20. And then it expects a very simple response back, but on the order of, okay, that has been applied. Or Nope, that's an invalid request or note there's an error in communication or that the right is overwritten by someone else.

Right. So a lot of it is simplification of, like you're saying that actually do this work. There's a whole bunch of stuff that needs to go on. Yeah. But this layer is an interface between two systems. We try to make it as simple and straightforward as possible. Yeah. And I can

[00:41:24] James Dice: imagine what you get from manufacturers who aren't used to communicating that information.

Right. Um, Upwards in the architecture And w when I've done this, right, there are multiple places to connect for a given command. Right? You might have a server level, like you said, head-end that has that set point. And then you have the zone level, lower level controller that has that set point as well.

So are you specifying,

whoever's listening to the audio on this right now. Trevor has his face in his hands and he's rubbing his eyes.

[00:42:03] Trevor Pering: One of my trigger words in there which is command. And let me explain a little bit about what, what that, why that's a problem, right? And this is the other thing that UDM does is it has a fairly strict what I call model based way of specifying things, right? A model based way says, look, I want this point to be 20.

So this point should be 20. It's very simple declarative statement around what the system should be. Right. A command is an exchange that can get really complicated. Right. Which is like, Hey, change it to 20. I don't really want it. No, no, it really changed the 20, but I had this problem, you know, all this back and forth.

Right. And so we, what we do is we impose this model based structure on things. And a lot of what it does is take that complexity that you just talked about, which is very real right. And it has to be there somewhere, but make sure it's cleanly, isolated and abstracted in one part of the system. Right? So maybe that's something that has to go on between the gateway device and over back that IP to some other device in the system.

Right. But we very careful don't expose that to the rest of the world, you know, contain it damage control. And that's a key concept that is actually one of those things that is really hard for people to really internalize with this because I'm sure man, Are very easy to understand they're very direct and how people do things, right?

The problem is they don't scale. And if you look at a thousand things, you have to deal with in a building, all these back and forth commands and protocols, you have to keep track of which messages made through where the errors are. All this other stuff. It's kind of a big nightmare, but if you just have a very, you know, reasonably simple model that says, look, this the punch would be 20.

Okay. That's a way to simplify it at that scope. Right? And so that's one of the other key differences that we do with this as a way to make it scalable and manageable for a lot of these studies. Yeah.

[00:44:02] James Dice: Yeah. I mean, there's a lot of, so I call this a and the listeners will know that I, I call this area of the industry advanced supervisory control and like what a lot of the cloud software providers are doing is doing that command and handling all of that complexity on their own.

Or they're just going, I hope this works and sending that. But what, what I'm getting at of you is this is really a it's a holistic system. Right. And someone has to manage that complexity somewhere. And I think what you're saying is you're saying if in order for us to qualify this device, we need to be able to do this sort of model based cloud controller right back in order for us to qualify your

[00:44:46] Trevor Pering: device on.

Yes. Yes. And a lot of it comes down to understanding you have fundamentally too. Different representations of the system, no matter what you think you have, you, you have to, in some ways, right? One of them is essentially as built or what the system is actually doing. Now, this is the physical thing that you have, someone programmed, something plugged, something in the wire was here.

That's what you have the other side then is the model of what you want to have or you think you have, right. And what this does is change it into a problem and making it very clear that the problem is look, how are these two models, representations of the system different? And when they're different, you have two solutions, right?

One is an error or alert of some kind that says, Hey, that device down there in the building is misbehaving. Somebody needs to go do something about it, right? Or it's the other way round the devices. Oh, I'm not complying to what the model says. I should be. I'm going to change my behavior or settings. I'm doing what the model tells me I should do.

Right. And it turns all of this complexity and stuff that you're talking about into a relatively simple differential comparison between these things and resolution of that. And that's one of the fundamentals that we think about in terms of automation and how do we deal with these, the systems of scale.

So are you saying though

[00:46:07] James Dice: that, that it's the device person who set up the devices responsibility to check the differential there and their devices, capabilities need to accommodate the ability to

[00:46:21] Trevor Pering: change? No, I mean, I'm saying that there is a difference and it's something on that side of the fence, their responsibility to change, right.

The differences, right. It could be that the device itself is smart enough to know how to change and adjust what it's doing. Right. Or it could be that a technician has to go in and do it. Right. We don't specify how it's done. We just specify this is what we want. And yeah. So it's exactly that though. And that is a key difference.

So in a lot of systems right now, right? Yeah. Someone has to walk out there and poke at things until it turns green and does the right thing, right? That's the truck roll that we want to avoid the fallout of this. We're trying to get to a world where that all of that stuff happens, but maybe you have, you know, in Jetson world, you have a robot going over there and doing it, or the device itself, if it's purely virtual changes, can, can do what it needs to do to conform to that model.

Right. And we're trying to get away from all these expectations around updating things or whatever it is requiring a person to do it because that simply doesn't scale. Right. Right. I got

[00:47:29] James Dice: to, I got to go out to the building. I gotta plug my specific laptop with the right license into the controller, but this specific court uh, anyway, makes total sense to me.

How, how where's this project at in the, you know, deployment, scaling up where's it at today?

[00:47:49] Trevor Pering: Oh yeah. And we have a reasonably large number of manufacturers who have implemented this and it's, you know, a lot of it is, like I said, very open standards. MQTT is a very common one out there that was developed by IBM quite a while ago.

And it's like, so, you know, a fair number of people have done that then, you know, getting into a certain Jason format is, is not hard either. And so we have, you know, a viable set of devices, you know, viable, meaning you can put a complete building together to do this. Right. Unfortunately I can't go into, you know, the details of who those things are.

Right. But it's perfectly sufficient to get a building up and going, and we're getting new ones in all the time. Right. Because what happens is some manufacturers like, oh, if I'm going to be on those projects, I better implement the standard. So they implement it. No, they go. Right. So, yeah, that's been successful at that, where it's getting tricky.

Right. When you start to get, so that's where basic operations connecting telemetry, ingestions, and the right back stuff. That's all reasonably straight forward where we're still growing, let's say, or, or working on things or some of the more complex features, like for more updates or just model updates and things like that, because those are super important, but they actually aren't as straightforward as you'd like them to be in some cases.

So we're still, you know, still those are not really there yet. And that's a place where we're, we're doing active planning and things to get to, but that's, that's the state, right? So when you talk about this overall concept, they're really basic stuff, telemetry, ingestion, and security and all those things.

That's really solid. And it's just, it's just the question of like, well, how much can you get out of the ecosystem in terms of these techniques? Got it, got it.

[00:49:25] James Dice: And what the firmware updates is. I mean, that scenario where you got to go roll a truck to do it today, right? I mean,

[00:49:32] Trevor Pering: and what pushback.

Yeah. And it's, it's a lot of there's a couple of things that come into play and it gets out to something else that you alluded to when you talked about, you know, getting my, you know, having a technician go out is from a security perspective, there's a lot of mistakes there that are made essentially that are totally unintentional.

People have good intentions, but you, you make mistakes. And that's where often security holes come in and compromises companies, because somebody left the door open or unlocked, right. Just by accident. And so part of it too is not only minimizing truck rolls because they're expensive, but because we want to get to a place where things are done in a very secure and regulated way.

Right. And the pushback really just his confidence in that. Devices can automatically update the firmware and everything will be okay. Right. And the thing is, if you look at every single iPhone out there and Android devices and web browser, and all, all of a sudden they're updating themselves all the time, right.

Those ecosystems do this. And so look, it's, it's doable. I, you know, no one can tell me it can't be done because I have enough examples of it working just fine. Right. It's just a matter of how do we get that into the industry and change these practices that people are used to that to make that build a confidence in it?

Totally.

[00:50:52] James Dice: Yeah. I think just my, my maybe cynical 2 cents there is that there are business models that are built on that truck roll.

[00:51:03] Trevor Pering: It

[00:51:04] James Dice: might be a little bit difficult to change those. When you start talking about, you know, 40, 50, 60% of a company's revenue is built on those types of tasks.

[00:51:13] Trevor Pering: Right. And it's, even though I'm a technologist and, you know, computer scientist by training all this other stuff. When I go into a meeting with a, a partner, right.

Some manufacturer, something about some of these things, one of the first things I try and understand is what is their inherent business model? Because I know that if I can't line up our technical objectives that are based off of what our business model is with what their business model is, it doesn't matter.

Right. Because yeah, you're exactly right. And that's actually one of the biggest challenges in all of this is how do you find the people that have a compatible. A compatible business model. Right? Yeah. And cause that it makes, makes or breaks any of these engagements around things.

[00:51:57] James Dice: Well, I think this is one of the reasons why I keep kind of going deeper with you guys because you guys are on a similar quest to what I'm on, which is like remove the unnecessary complexity.

Right. And I think my opinion at night take way too long for this to happen, but the companies that like want to maintain the complexity and again, no one's doing this on purpose. This is not a malicious thing. But if a business model wants to maintain the complexity, it will do so until, you know, a competing business model wins out.

Right. And I think that's, that's the kind of story I'm trying to tell here is that these staff are trying to get removed from the system.

[00:52:36] Trevor Pering: Yeah. And you know, it's a classic case of what the business world is called disruptive innovation and disruptive technologies. Right. Is, and it's well understood.

Probably. Just of how new technologies and capabilities have to really find their voice and find what really new capabilities they bring. That just can't be done with the old model, because if you're doing something that competes too directly with the old model of stuff, you're never going to get over that barrier of, of really why are we doing this new thing when this other thing can be evolved to what we need?

So it's a, it's a really tricky challenge that, that faces just technology in general is innovation comes along. Right. And where

[00:53:19] James Dice: do you think this industry, you know, you're, you're kind of a relative outsider a little bit, you know, kind of putting your, dipping your toe in a little bit here uh, maybe a lot of bit, right.

Or where are we at in that sort of disruption?

[00:53:33] Trevor Pering: Arc. Jeez, you know, I feel like. No, we're on the edge of a cliff is the right word, you know, the right almost there, but we've almost also almost right. Been there for, for quite a while now. Right. I feel like there's a whole bunch of different pieces that need to line up that are starting to line up and are going to be there very soon.

Right. And you, you can look at many different indicators of this and, and I've heard this in some of the other podcasts you have, right? It's like, look, we, we, we have successes out there. We're getting a deeper understanding of what's going on. We're figuring out some of the security challenges we're making progress on this other front.

And at some point, you know, the, the architecture that we have when we think about these systems, I think I charted it out in my head and Sabina talked about this a little bit, her podcast, which is, there's like 10 different layers to this thing. All of those are like, we have clicking in different projects.

And so over here, we have this set over here. We have this set and. No, because it's too hard to do them all at once. But soon we're going to have some places where they all come together. Right. And that's when you get this flip, the switch and like, Hey, we got this new stuff and I'm pretty sure once you have that thing that people can point to and say, oh, there's a success that we're looking like, then it catches.

And that's how you established whatever that new thing is. So I love it.

[00:54:57] James Dice: Well, cool. This has been fun. Any other, besides those three projects, any other talking points?

[00:55:03] Trevor Pering: you know, when you look at how we approach this, right, there's two main overriding principles that the two factor into all of this stuff, and I'm just hammering these home.

Right? Cause these are really at the core of everything we think about. And one of them is a security principle called defense in depth. Right. If you look at what we're doing. You know, we are essentially trying to fix the same security problem in three or four different ways, right? It's not good enough just to do one, right.

Doc with qualification tries to make sure the devices implement this thing, right? Not in the network. Stuff is preventing devices from talking. You know, in case one is compromised and it can't talk you DMI. And the way it's structured is using very strong cloud based authentication to prevent some other kinds of compromises.

All of those things, each one of them itself would be good enough to, to solve the problem. If it was done correctly, the problem is you need to make it really robust. And that's why this concept of defense in depth really plays into that. The other one, which is talked about this before, right, which is automation, and to do this at scale, we have to have tools and capabilities that automate, and this is a principled way of doing design.

Not something you can bolt onto systems afterwards in a lot of cases. And so when we look at all of this, and this is very endemic to Google, and Keith mentioned this in some of his, you know, his ponderings about this, which is doing things at scale really just comes down to automation. And that's a large part of all of these projects that we look at is really coming down to w how, what does this mean for automation?

How does this automate things? How, how do we deal with this? So brilliant.

[00:56:43] James Dice: What was the defense layers of defense? What would you say? What was that one?

[00:56:47] Trevor Pering: Oh, it was defense in depth, defense in depth. Got it.

[00:56:51] James Dice: My is like coming up with sports analogies in the

[00:56:54] Trevor Pering: background. Cool. Likewise zone coverage. It's you know?

Yeah,

[00:57:00] James Dice: yeah. I was, my, my mind is a soccer mind, so I was thinking of different formations, but anyway uh, well, let's play church's two truths and a lie. Kind of a fun way to close out. We've been doing, are you, are you prepared to stump me?

[00:57:16] Trevor Pering: I I'm. I'm hoping I'm prepared. Right? I don't know. I've heard you've had a good track record, but you're not perfect.

So we'll see. So first one of them is that I have a really bad sense of direction. I get lost pretty easily and that's just, it's weird. Second one. And this is also very weird. I still count on my fingers. I never grew out of that phase of life and I'm still using my fingers to count all the time.

The third one is that I am colorblind and it's still amazing how many times I have to. Pipe up in a presentation sometimes with the VP around going, yeah, that's really not a good choice of colors. I just can't. I do not know what you're doing. So those are my three things are all pretty basic. Um, See if you can uh, tease this one out.

[00:58:06] James Dice: I think it's going to go with my gut here. The first one's a lie because you strike me as someone that would use navigation app. I can't think of which one you've used, but that's, that's, that's what I'm going with. So that's your

[00:58:23] Trevor Pering: final answer.

So, um, yeah, you're right. Um, I actually have a really good sense of direction. I just didn't smell that one. Well, you know, I have to relate this back to though, and I was thinking about, you know, and I know this is geeking out here, but it's actually one of the reasons I do really well with computers. One is computers count really well.

And so, yeah, I still count on my fingers, but that's partly why I'm good at computers because I let them do the counting tip of colorblindness. It really doesn't matter. There's no, you know, most of the time in computer world, it doesn't matter. And I can set the color scheme to whatever I want. And the thing is the sense of direction.

It's not so much the navigation app or any of that stuff. That's that, that part of your thing, isn't that's whatever, but it's actually, you got lucky on that. Yeah. Yeah. Well, it's, it's really that it is a spatial awareness and it actually helps me a lot with these computer systems. Right. Cause they're really.

And understanding how all these different pieces connect together with a network, a qualification, and all those things jumble up in my head in a way that is very similar to how I feel when I'm out there in the world moving around. Right. So, yep. You called it right on that one. So good job.

[00:59:34] James Dice: Totally. I think I have that as well.

Like I always know what direction I'm facing except for, in one building in the world. I don't know. Whenever I walk into it and that is Denver international airport. I have no idea what they're even there's like east and west terminals. So that helps, but like, I still don't know where I'm at, but one building that's dumps me cause they turned me around so many times.

Well, thanks Trevor. This has been so much fun. I love the series 1, 2, 3. We have with Google going. I think I know who I want to have next from the team. Uh, We'll have to convince her to come on. So, uh, thanks so much. And yeah, talk to you soon.

[01:00:12] Trevor Pering: All right. Thank you.

[01:00:17] James Dice: All right friends, thanks for listening to this episode of the Nexus Podcast. For more episodes like this and to get the weekly Nexus Newsletter, which by the way, readers have said is the best way to stay up to date on the future of the smart building industry, please subscribe at nexuslabs.online. You can find the show notes for this conversation there as well. Have a great day.