In this insightful discussion, the Collab Collective's Craig Durr interviews Jeff Smith, Head of Product - Workplace AI, Meetings, and Spaces at Zoom. The conversation delves into how Zoom is leveraging AI to redefine workplace collaboration. From advanced generative AI applications to the emerging capabilities of agentic AI, Jeff provides a glimpse into Zoom’s innovation in optimizing workspaces, enhancing productivity, and creating frictionless user experiences.
Or tune in on your preferred streaming platform:
Craig Durr: Hey everyone, this is Craig Durr, Chief Analyst and Founder of the Collab Collective, and I want to welcome you to a really special conversation that we are having about the workplace and AI. To talk me through this conversation, we have Jeff Smith from Zoom. He's head of product for meetings, spaces, and workplace AI, right?
You know what's incredible about this is all that information we want to talk about here is in the context of what takes place in the office, right? You've been working on AI for many years, right? It's almost been an evolution that I can look at the products and the features that you guys have brought forward as well. Let's talk through the evolution of AI in the Zoom experience.
Jeff Smith: Yeah, if we go back four or five years, the main problem that we were trying to solve was this concept of meeting equity. So you thought about people that were remote, people that were in conference rooms, and they didn't feel like they were equal participants in that kind of meeting.
And one of the main challenges that we had was on the video side, separating out that one stream of video into “I've got Craig over here. I've got Jeff over here.” and being able to recognize those people as individuals.
Same problem on the audio side. First, you have to capture everybody. You know, great acoustics and all that, and make sure that you've got the hardware that you need to do that effectively. But then, it's just still just one audio stream. So I have to be able to separate that out into “This is Craig's voice. This is Jeff's voice.”
And now, when you get into that meeting space, I've got individuals that are recognized as themselves throughout that whole experience. Now, why is that important when we think about the evolution of AI? It's foundational, so if I don't have a transcript where your speech is recognized from you and my speech is recognized from me, that transcript becomes garbage. It's a conference room that's speaking.
Craig Durr: It’s just like that same image, like you said, if there's four or five people in one image, I don't know if that was Jeff, that was Craig, that was whoever else was speaking in the room right now. You guys actually brought this forward in some really powerful features–Smart Name Tags is one of them. The other one is the Intelligent Director feature, right? So you've proven that these are not only challenges that you can solve with AI, but you've done it in a very elegant way that sets up this next evolution of your AI experience.
Jeff Smith: That's right. So we started with Smart Gallery, and it was just on the video side, being able to put people into individuals, just so that they could be seen effectively. So I could have eye contact, and I could see people's expressions and all that–very important. Being able to expand that to a large room where I can't see that effectively from a single camera angle, Intelligent Director becomes really critical. Because I need all of the spaces, not just a small conference room, to be covered in order for the whole power of generative AI to become pervasive.
All your meetings now can have an AI assistant. It could be in person. Just the two of us walk into a conference room and just have a chat with each other. AI Companion can take notes for us, so we don't have to sit there and bother with writing notes, taking action items. AI Companion can do that. If we're going to meet in a coffee shop, I've got AI Companion on my mobile phone, so I can put it on the desk, no problem taking notes, taking action items–making sure we stay on track on the agenda. All of those things, no matter where I'm having my meeting, in person with remote people, all of it can be assisted by AI.
Craig Durr: So this is the evolution that we're talking about, kind of going from these applied AI experiences around the video and audio, leveraging that as a foundation to create these generative AI experiences. But now, we're getting to a really exciting area of agentic AI. I'm sure you're thinking about this in the same foundational sense and building upon these previous steps. Share with me your thought process behind that.
Jeff Smith: That's right. So generative AI in taking that unstructured information and putting structure around it, generating tasks… that's really the beauty of generative AI. Now, when we look at agentic AI, it's the ability for my assistant to now take more action on my behalf, to become my executive assistant, not just my intern assistant. And those types of tasks that AI Companion can do for me become more and more powerful and save me more and more time. That's what we're really focused on right now.
When I think about Spaces, when I start my day, I have a lot of decisions that I could be making, and it could be making decisions about where I should work that day. Should I be in the office? Should I be remote? But I need a lot of information in order to make those decisions effectively. And AI is fantastic at taking large amounts of data from different data sources and being able to synthesize that and give me good direction, or even just tell me what to do.
Craig Durr: I mean, you actually have this treasure trove of data within the workplace experience, right? You're not only talking about email, you're talking about calendar access, you're also talking about the idea of seeing my colleagues and where they're going to be. So predictively, it could probably help me understand when is a great time to go into the office.
Jeff Smith: Absolutely. So some of the pieces of information that you might not look for is… there's a meeting about this topic that you really care about with this person that you think is really important, and that person is going to be in the office during that meeting in building five, and it would take you an hour to figure that out. But because my AI Companion knows that this is my VIP List and these are topics that I'm following and working on, it can say, “This is a really important day for you to be in the office. You can be really, really effective if you are in this conference room on this day at this hour.”
Craig Durr: I love it. So it is taking that predictive, proactive stance on my behalf, right? And it's pulling across all these data sources, as you say, could take me hours to try and figure it out, right? This is the difference from being just efficient to also being effective in terms of my work day.
Jeff Smith: That's right, even simple things, like, “I am in the office today, and there's somebody else in the meeting that I'm attending that's in the office, and we booked two different conference rooms.” It’s a simple problem. But last year, we would have shown up into those conference rooms, we would have seen each other on video and gone, “Well, that wasn't very smart.” And then one of us moves to the other. You've lost five minutes. So instead, AI Companion could reschedule those on our behalf. The meeting that you know is in this conference room, and that's the only conference room that's booked for the meeting because AI Companion just took care of it for us.
Craig Durr: Let's lean into another stakeholder in this process here. So we're talking about these wonderful end user experiences. But you also have to think about what AI and agentic AI can do for facilities, for IT administrators, especially as they're thinking about the optimization and the utilization of the workplace itself.
Jeff Smith: Yeah, AI is built for consuming large amounts of data and being able to distill insights So for the facilities manager, this becomes really important. I can understand which spaces are being utilized, which spaces are not being utilized, what characteristics and what patterns are existing in those highly utilized rooms versus the rooms that are less utilized. So I can start to say, “My users are gravitating, and they're telling me something through this data.”
But AI Companion can distill that immediately and say, “You should design more spaces that look like this because these are the spaces that people are gravitating to, and they have these characteristics.” So that's the kind of proactive insight that I can get instead of just waiting for an alert from the system that says, “The camera in Conference Room B is down.” Then, I can be reactive, and I can go send somebody to fix that camera. Or we can look at the video feeds, and AI Companion can tell me, “This is an awkward room because people keep walking around it like this. And this cable is bound to get kicked at some point.”
Craig Durr: All right, you are painting a really exciting picture of the future. What I hear you talking about is how agentic AI can make the end user experience more frictionless, and you're able to help out IT administrators, workplace administrators… There's a lot of power here that Zoom is building a great foundation for to bring forward, right? If our audience wants to learn more about how Zoom and its AI Companion can help in the workplace, where can they go?
Jeff Smith: Oh, absolutely. So go to zoom.com/rooms. We've got all your information there.
I appreciate it, Craig. This is great.
Craig Durr: Hey, thank you so much for the insights. This has been a wonderful conversation.
Everyone, this is Craig Durr with the Collab Collective.
Jeff, this has been wonderful. We've learned a lot about AI in the workplace.
Jeff Smith: Well, I hope so.
Craig Durr: Good. Take care, everyone. We'll see you later.