Broadcast Exchange: A dynamic virtual environment for BBC Olympics coverage

BBC’s coverage of the Winter Olympics in Beijing utilized the latest in virtual studio technology to transport presenters and viewers to a snowy ski resort.

Using the Unreal Engine paired with the latest technology from Vizrt, the project’s designers digitally sculpted mountains for the broadcast environment while carefully thinking through how to keep the visuals dynamic with data and highlights.

Jim Mann of Lightwell and Toby Kalitowski of BK Design Projects join the Broadcast Exchange to take us inside the virtual set design, the development in Epic Game’s Unreal Engine and the larger impact of the technology on the future of storytelling.

Listen here, or on your favorite platform.

Subscribe and Listen

Video: YouTube

Audio: Apple Podcasts | Spotify | TuneIn | Pocket Casts | Amazon Music

Show links

Transcript

The transcript below is in unabridged form.

Dak: Hello, Broadcast Exchange from NewscastStudio. I’m your host Dak Dillon. The Exchange features interviews with people who are shaping the future broadcast design, technology, content, and more.  Today, I’m joined by Jim Mann of Lightwell and Toby Kalitowski of BK Design Projects. The team recently completed BBC’s virtual environment for the Winter Olympics in Beijing. Vizrt and the Unreal Engine were paired to create a dynamic studio. The team even built some mountains.

Dak: Thanks for joining me today to talk about the BBC’s unique presentation of the Olympics. You’ve learned a lot since you created the Tokyo studios, which also relied on this heavily virtual production pipeline. Walk us through those lessons, and then how that’s set up where we are today, with Beijing.

Jim Mann: Lessons. Yeah, it’s a funny one because it’s not just lessons, I guess, from Tokyo. I think it’s also lessons from Pres 2, which is the in-house studio that we worked on for the BBC, which launched a few months prior to Tokyo. In essence, the Winter Olympics studio, it’s that studio.

Jim Mann: It’s from the same green-screen studio, and it’s a modified version of that virtual set. Whereas previously we were using, or the BBC have been using, a main presenting space with windows, and then beyond the windows, it’s a photographic environment.

Jim Mann: Having seen the potential of what could be done on Tokyo, well, we thought, “Well, why not? You know, let’s go for it. Let’s have a bit of fun with this one,” and decided to try out the landscape in [inaudible 00:01:46]Unreal. We started looking at this… it was actually before Tokyo, wasn’t it?

Toby Kalitowski: Yes.

Jim Mann: Am I right, Toby?

Toby Kalitowski, March, it was March. March or April last year. I think the biggest thing in terms of lessons learned from Tokyo was the multi-level approach, where you’re offering the editorial team various different positions they can use for different program strands.

Toby Kalitowski, Tokyo: It was amazing. That sense of directors being able explore and find new spaces for their programs was a joy to everyone. That’s kind of what was at the core of this idea of creating multiple levels and different presentation spaces outside, as well as inside.

Jim Mann: I’m a huge fan of serendipity. I think there’s a lot of that in there, but I also like to think that serendipity is only going to happen if you’ve got some decent design ideas, and a decent context in terms of the design to start with. It’s about just extending on what we’ve done before.

Jim Mann: It’s a team we’ve worked with previously on quite a few jobs now, and we’ve got to know them quite well. So, we tend to know what they like, and we tend to know what they don’t like. I think that helps as a designer, when you’ve got a better idea of what will appeal to the client, and what the client can make use of.

Jim Mann: I think what’s really happened with the Beijing Winter Olympics studio is that the BBC’s taken ownership of it. They’re doing things that we thought would be good, but they’re taking it just that stage further.

Jim Mann: I guess one good example of that is, there’s a seating position where, when we were putting together packs of presentation images for the designers, in its initial stages, we were spinning around the Unreal model. We had a look back at the building from within the landscape, and immediately, it’s like, “Yeah, this is interesting. Can we make use of this?”

Jim Mann: So, of course being a design presentation, we dropped in a 3D figure, set up a fairly standard mid-shot with the building in the background out of focus, but getting that sense of a presenter in the landscape, and thinking, “Oh, this will be sort of top of the show. You know, they’ll do a quick announcement there, then move inside.”

Jim Mann: That then developed, you can tell they were interested because they then said, “Oh, well, can we have a floor?” Because the presenter was floating in the sky. We built a little hill for them. Of course, if it’s just snow, that’s not very interesting. To break up the landscape, you can put some rocks in.

Jim Mann: The next thing you know is, you’re being asked if they can sit on the rocks. So, you put in a rock that they can sit on, and then it’s, “Well, could there be a bench?” We had a little bar area, so we pulled a bench across. The next thing we know, they’re borrowing stools from elsewhere in the BBC, covering them with leftover bits of green screen.

Jim Mann: Hey Presto, there’s a brand-new presentation space called “the bench”. It’s an informal space and it just works, but it’s working because we’ve done all the hard work in the landscape and the rest of the design. I think those kind of things have been what’s really rewarding about this particular project.

Dak: Now, you mentioned Studio Pres 2. Dak: Now, you mentioned Studio Pres 2.

Toby Kalitowski said: It was a studio that was lost and never used. They wanted a green-screen, generic environment. We created a multilevel environment for them. It was our intention from the beginning that this environment would have areas that we wouldn’t even consider using in the next year or after.

Toby Kalitowski : It would be an ever-changing scenario. The BBC was very supportive of this idea. It’s been very successful. It’s been massively overbooked, so much so that it’s difficult to get into the studio to do testing, and all the rest of it. So it’s slightly… we’ve been a victim of our own success in that respect. It’s been quite hard to get time to test new environments in there, but that formed the basis of this new studio.

Dak: You can take what you learned from Pres 2 and create these sleek white columns. Then, you will have an embedded fireplace. Finally, you will have these breathtaking panoramic mountain scenes. Let’s take a look at the design of the Olympics.

Toby Kalitowski, The architecture was there. You said that it was a modern, sleek environment. We wanted something warm so we added the fireplace. There were some things that we had to do in the actual set. We added ice on the sunken floor areas.

Toby Kalitowski : The landscape was the real focus of the entire project. That’s where we had an amazing time. Jim was amazing and Unreal was awesome. We learned so many things. Yes, it was the main thrust of the whole event to create a 3D environment, a giant mapping. Then, we set our presentation decks out in the snow.

Toby Kalitowski: We discussed things like, should they be wearing quilted jackets, and how far… I think what’s interesting about the whole thing is, you’ve got to take it with a pinch of salt because we’re not yet in the world of photorealism.

Toby Kalitowski: It’s almost… We have people in Tokyo who thought we were in Tokyo. That was something we never imagined. I have had people say to me, “Are they out in Beijing?” I’m like, “Have you actually seen Beijing?” It doesn’t look anything like this lovely, alpine, snowy background.

Toby Kalitowski, So it was an exploration. The idea of using Unreal in order to create a landscape. It was a risky venture on our part as there was no guarantee of any results. It was a, “Hey guys, what do you think,” to this. It’s probably fair to say that there’s probably a bit of our inner [inaudible 00:08:30] coming into it, in that it’s almost like a Bond baddie embedded in the mountain scape.

Toby Kalitowski: It was having a sense of fun and having a play, a “what if”. “What if we did this, what if we tried that? Wouldn’t it be cool if we did this?” I think that’s where a tool like Unreal is fantastic, because we were able to just develop that and play with that, and really have a lot of fun with that kind of idea.

Toby Kalitowski: At the end of the day, there’s ski tracks in front of the building, all the paraphernalia that you might find around a ski resort, but at the end of the day it was, “Let’s have fun with this.” That’s one of the nice things about working with sport, TV sport, is that it can be quite playful.

Dak: You’ve mentioned Unreal many times. This is, I believe you’ve told me before, one of your first big projects where you’ve done most of the design and development straight in Unreal, and not necessarily in Cinema 4D or another piece of software first. How has this changed your workflows and the way you conceptualized these spaces?

Jim Mann: Massively, massively, because Tokyo is my example. Tokyo started out, it was, Toby would build a model, turn it to me, I’d do something, send it back to him. It was like playing tennis, with models going forwards and backwards. [inaudible 00:10:02] packages, which it’s not ideal. It was simply a matter of exchanging ideas.

Jim Mann: Next, we created an animation for our pitch presentation in V-Ray. This meant that parts of the model had to be completed several days ahead of time, and that spare machines were being used for a few days just to render all the frames.

Jim Mann: With this Olympics, because we just thought, “Yeah, whatever, let’s just go for it.” We’ve got the main space already, or we’ve got the shell already in Unreal. We just started to add to it in Unreal, and then we began to take bits into Unreal. Because we have the power to see and move around, it was all about exploring and finding out what works. It was a great idea. It helped as we were able to borrow quite a lot of things. However, we also saw the potential and possibilities.

Jim Mann: We were able bring that in. Not just as a suggestion but as something more credible and viable. When you’d think, “Well, what if we had some decking?” So, you add some decking, and then you add a snow particle system, and all those things start coming together. It’s like the picture really starts to evolve.

Jim Mann: It’s really fluid. You’re not spending your time waiting for a render to finish, which is the most depressing thing ever now. If we had a presentation, we’d then, in a fairly short space of time, put together a short animated sequence. Being television, that showing the client something, as a volume in which a camera can move, that’s gold. This is truly gold.

Toby Kalitowski, a designer: Typically, towards the end of the design process, we would spend a good part of a day or even an entire afternoon on Zoom. We would be in Unreal. Jim would be moving around, and it’s just incredible. Jim would be painting snow and we’d be saying, “Okay, let’s lift the mountain here and, okay, let’s paint some trees.”

Toby Kalitowski: Obviously, there were lots and lessons to learn about resolution of things, paring stuff back, and optimization, but actually, the process of designing, it’s so incredibly fluid. It’s impossible to imagine going back to where we were a year ago. Waiting for someone to render something is madness. So, it’s been really exciting, really thrilling, I think.

Dak: Finally, the final product was integrated to Vizrt, Viz Engine 4 and a variety other Vizrt tool sets. Talk about this process and how it all worked together.

Jim Mann: The great thing about working with a client like the BBC is, they’ve been using Viz for forever, and there’s a good knowledge base within the BBC. It’s a really good team of people working in the graphics department at the BBC.

Jim Mann: Andy Bowker was our main point of contact. In a similar way to myself and Toby passing models backwards and forward at the beginning of the process, towards the end, it’s myself and Andy passing copies of the Unreal project backwards and forwards, and then making adjustments.

Jim Mann: I think what’s been really good this time is, Andy’s also been involved in the Pres 2 work. Andy is not only an expert in the studio, but he also has a deep understanding of this project and this design. So, he’s been able to suggest things along the way.

Jim Mann: Andy is responsible to all the penguins and deer and the polar bear. He’s been able to take on that whole process of taking the Unreal project and doing the integration with Viz. He also interacts with the virtual set operators in the gallery at broadcast times, which is another responsibility.

Dak: They have a lot of virtual screens inside the studio, so they can bring up highlights or queue up graphics. Talk about the challenges of integrating those into a virtual space, and making sure the presenters or the talent can fully utilize and understand, even, what they’re looking at.

Toby Kalitowski – For Pres 2, we designed a projector that projects an image onto the green. [psych 00:14:55]. We have a standing position that is used for all the setups during the Winter Olympics. It’s a very, very small area of the set, to one side of the studio, where we have two standing presentation positions.

Toby Kalitowski : Between them, we have our projection area. That is how you can determine your position within the virtual space. Obviously, sometimes we didn’t do that. You will need a floor monitor to give you an eye line.

Toby Kalitowski, Actual design of the screen. You wanted to make them feel as real as possible. We ended up making the screens quite chunky rather than the thin, lightweight, bezel screens that you could have. They were just too fragile to be believable as outdoor screens. So we spent some time creating the actual virtual assets.

Jim Mann: It’s almost as if things have to look not as good as they could be. It’s almost like you have to make the designs look less credible or more believable in order to increase credibility. There’s also areas where we’ve got the BBC logo on the floor, but we’ve scuffed it, and scratched it, and damaged it, and just made it look like people have been walking over it for a week or two.

Jim Mann: If you’ve seen that video of Aimee Fuller practicing her snowboard in the studio, you could probably understand how that might get damaged like that. It all adds to that sense of, I guess, it’s care-worn.

Toby Kalitowski: We had many conversations about that very thing, about beliefability, and that fine balanced, the uncanny Valley thing. The more you get closer, the more strange things can be. It’s quite a hard area to make sure you are in a sweet spot.

Dak: I think I just answered the question. We are now seeing two approaches to virtual studios. There are two types of virtual studios: the completely abstract and those that are more grounded in reality. The ones that often fail are those weird ones that are somewhere in the middle, where it’s like, “Oh, these are the correct-size mullions on a window,” but then suddenly the ceiling is 300 feet tall.

Dak: I love this space. It feels like a real ski lodge. It is like something you might find at the top Canada’s gondola. It feels very real. However, some studios out there, like you mentioned, have the uncanny valley effect. Are they a spaceship or studio? Where exactly are they? What’s that sense of place beyond just the snow behind them?

Jim Mann: If a viewer is looking at something and trying to think, trying to decide, “What is it? What’s it meant to be,” then on a certain level, we’ve failed. Whereas I think with the Olympic studio, I don’t think… there’s no guessing. There might be a sense of, “Where are they?” Which is very different to, “What is it?”

Jim Mann: The “Where are they” is a much more interesting conundrum to have. There is an element to that. I suspect that probably comes… my background is I’m a registered architect. Architecture wasn’t very interesting, I moved onto different things, which I’m so much more engaged in, but I’ve still got that sense of, I know how big a mullion is. I know how high the ceiling would be.

Jim Mann: If you’ve got a ceiling any higher than that, well then, where’re you getting that glass from? You can’t get glass panels… and there’s all those limiters in my subconscious. We can’t do anything too ridiculous. It’s almost like we’re saying to ourself, “Well, how would it be built? What would it be made from? What would the detailing look? What would the joints be?”

Toby Kalitowski says: Every element we deal is true-to life scale. Therefore, every step, each handrail, and every beam are justified. If it’s a cantilever, it has to be thicker than… it has to do its job. I think you’re right. There are lots of sets out there which have a theatricality to them, and they don’t tie themselves to those structural rules, really.

Toby Kalitowski: They may look great, but there’s something about it where you just think, “I can’t invest in it. I can’t believe in it,” and the person standing in it doesn’t feel like they’re a part of that scene. Yes, architecture is the source of all our inspiration, and all our excitement. It’s an amazing place at the moment, to be able to play, in such an environmental way, creating spaces that are more like buildings than they are like sets.

Toby Kalitowski: I don’t feel, really, we’re set-designing anymore. I think we’re designing environments, but it is interesting how out there, people still are doing… there’s fourth-wall, quite stagey-feeling virtual sets where you just think, it’s almost like a proscenium approach to theatricality. Which is bizarre, because we can do anything, and yet we’re stuck in a Victorian theater sometimes.

Dak: It has to be precise with all details. Or it can be abstract and just have floating glass in an abstract space. Because, then, you know it’s fake and you don’t conceptualize that in your brain as being real, but even in Beijing, even the snow is fake for this Olympics.

Dak: It’s all a little bit man-made. What challenges remain in virtual production, and what are you doing?

Jim Mann: I think it might be performance issues. Obviously, creating a virtual environment as opposed to a virtual set, it’s much more expansive. Unreal is great at handling those large data sets, but I’m still getting used to the idea that you can’t populate every single part that you see for every shot, with the highest level of detail.

Toby Kalitowski: For me, it’s all about lighting, the quality of lighting, and the quality of contact of the presenter within the virtual space. I want to see amazing contact shadows and fantastic reflections. That’s limiting for us, specifically. I know Viz are developing that, and that’s obviously just around the corner, so there’s lots just around the corner, but the closer you get, the further you feel you are, in a way.

Jim Mann: The contact shadows and contact reflections, it’s coming in now. Although it might have been possible to do more at the Olympics with that, this brings up the issue with the Pres 2 studio’s popularity. The studio was so busy prior to the Olympics with the Australian Open tennis.

Jim Mann: So, there wasn’t any opportunity for the guys at the BBC to actually test those features properly, to get them working properly. It’s integrating the presenter with the virtual set. A lot of that comes through… it’s either shadows or it’s reflections. If you’ve got a real floor, that’s great, but a real floor can be limiting.

Jim Mann: The flexibility of whether they’re on ice or wood, or on snow, is something you have to compromise on. Those are the three different virtual floors we’ve got at the moment. If we had just one floor material, that’s a magic carpet that’s traveling around this virtual world, which they are constantly stood on.

Toby Kalitowski : To me, it would be very interesting to have complete control over the lighting in the virtual space. This is possible from the gallery. It will allow us to work more closely together with the lighting director. I’ve had tantalizing experiences of the possibilities of doing that, but haven’t fully embraced that yet.

Toby Kalitowski says: This seems like a no-brainer. To have full control over both virtual and physical lighting, this is essential. Of course, that’s money. Ideally, you want moving lights, you can adjust all the color temperatures on the fly, and it’s all preset. You don’t have to do it, but people do it.

Jim Mann: Everything’s making a demand on that graphics card, and it all starts and finishes at the graphics card. As graphics cards get bigger, the software and the designers, we’re trying stuff more through the graphics card. So, we’re always going to need more.

Jim Mann: I’ve been doing this now since, blimey, I think my first 3D with a render was 1995, an AccuRender with Pentium 120s with like 128, not gigabyte, megabytes of RAM.

Jim Mann: You were always trying cut down the polygons, and cut down the lights. And then wait a few more hours for something pretty awful that at the time felt amazing. [inaudible 00:24:22] render. It feels a lot like that again, but so many generations on, and it’s exciting. It’s exciting.

Dak: In terms of the R&D, you’ve talked a lot about just that process. Is it a process that is longer than a traditional scenic load-in and training, or is it something that it’s still going to be a little quicker, just getting the set up and running on the virtual side?

Jim Mann: I think when the hardware’s there, when the studio setup’s there, it’s probably pretty quick. We could probably turn it around and get the file loaded in a very short time. As to how many day that would be, I don’t know, but everything depends on the complexity.

Jim Mann: If you’ve already thought about it properly, you already understand the limits of the physical space, and how that physical space relates to the virtual set, and vice versa.

Jim Mann: You already know where your talent should be within this design. You already know where you expect the cameras to be, and where the cameras can’t go. This should get more fluent and faster, but again, as things get more complex, then we’ll want to do more complex things. So, [inaudible 00:25:39] again.

Toby Kalitowski: It’s a conversation we had with John Murphy, who’s head of graphics, creative head of graphics at BBC, and we were laughing and saying, “I think people… the general assumption is, if it’s green screen, it’s kind of easy, because there’s nothing there and you’ve just got to plug it in and play.”

Toby Kalitowski: But actually, in terms of time and cost, currently, as it stands, it’s more expensive than a comparable physical set install, in terms of the time setting it all up. It’s very difficult, because you can’t compare it anymore. Because obviously, we can’t build what we are building, physically. It’s not like we could even attempt to do it.

Toby Kalitowski – The physical elements are now much simpler and your budget and time can be spent on the virtual. I think it’s generally costing more, I would say, at the moment, but that’s because there’s so much of ironing out of issues. But I think every production’s having that. I don’t think anyone has it easy.

Toby Kalitowski: I don’t think anyone’s cracked it, and they’re just breezing through stuff. I think everyone is having to deal with the latest versions, and graphics cards, and “Oh, there’s an issue with the [inaudible 00:26:54] box and the swop out, and do that.” Do you know what I mean? It’s constant stuff that’s happening.

Dak: To wrap up, what’s next for what you all are working on?

Toby Kalitowski: We’re currently working on a pitch for the Commonwealth games, which takes place in Birmingham this summer. I’m doing a lot of work at Bloomberg in the UK, who are expanding their studio facilities here.

Jim Mann: I’m currently working my way through every regional studio for ABC Australia. So, yeah, it’s February in England, the weather’s miserable. I spend my day admiring photographs of stunning sunsets in Tasmania or Hobart. Yeah, it’s just to make me feel worse. If any broadcasters out there would like a studio designed for the World Cup, we’re ready.

Dak: Sports never stop. There’s always that next event. I’m sure soon you’ll have to start thinking about another Olympiad only two short years away. Thank you for listening to Broadcast Exchange. Subscribe to the Broadcast Exchange on your favorite podcast platform or view our video episodes on YouTube.