Cutting Through the BS if Corporate Video! Episode 7 | The Many Faces and Formats of Business Video

Here we are again with Episode 7 in our series on “Cutting Through the BS of Corporate Video“!

In this episode, Jeff Sengpiehl, The Post Doctor, and I look at the two ends of the Enterprise Video Content Factory: the front door, where content comes in, and the loading dock, where it goes out. Our guest is Sarah Semlear, who runs go-to-market for Hiscale, a company whose pitch is that transcoding should be boring.

Every camera, phone and platform wants video in a different format. We talk through what enterprise video transcoding actually does for a corporate team: verified ingest, editing while a recording is still running, elastic cloud scaling that tracks real demand, and where a transcoder stops and a MAM or orchestration layer takes over. This is part one. Part two is a demo.

Watch, then tell us on LinkedIn: where does video get stuck in your organization, on the way in or on the way out?

Takeaways

• Transcoding is the math behind the factory. In Sarah’s experience, when a MAM chokes on a large backlog, the transcoder is often the real bottleneck.
• Ingest has to work for non-technical people. One defined action for the camera operator, and hot folders for everyone else, gets footage into the right places in the right formats.
• Verified ingest protects against two costly mistakes: wiping a camera card before a corrupted copy is caught, and trusting an upload that only reached the local cache.
• Growing files let a team clip and edit a live internal event while it is still recording.
• Elastic enterprise video transcoding scales up for spikes and archive migrations and back down when idle, so cloud spend follows actual demand.
• Hybrid done right means a stable on-premises baseline with cloud capacity for peaks, flowing back and forth as demand changes.
• Codec licensing is a hidden exposure. Some free transcoders do not include the licenses for the formats they encode.
• A transcoder should transform files. Moving, sharing and tracking them belongs to the MAM and the orchestration layer. Specialized parts that connect beat one box that claims to do everything.

Full Transcript

The Front Door and the Loading Dock (0:07)

Jeff: So far on Cutting Through the BS, we’ve defined the content factory. We’ve looked at the orchestration layer that runs the whole thing, and we’re going to get into the editing services soon. Today, we want to look at the two ends of the factory: the front door and the loading dock. How content gets in, ingest, and how it gets out, transcoding and delivery. And to make it clear, transcoding video is transforming it into a version that you can use in your factory.

Our guest is from Hiscale, whose whole pitch is a line that I love: transcoding is boring, think about it less. They build elastic, responsive media processing that scales up when you need it and back down when you don’t. And it runs on-prem, in the cloud, or hybrid. That last part, hybrid in a corporate IT-controlled environment, is a big part of what I want to get into today.

Eivind: So, fair warning to everyone, this episode could get a little technical. We’re going to try not to get too technical, but it might. That doesn’t mean that what we’re about to talk about isn’t important when it comes to enterprise video. We’ve talked earlier about sprawl: video files take on a life of their own and live on memory sticks and computers and tablets and devices and hard disks across the known universe. Well, guess what? All these different cameras and phones and applications that we use to create these video files all want to do things slightly differently. And when we want someone else to look at what we made, Facebook wants it in one format, watching on a big screen requires something else, and sending a big file across a slow internet connection just isn’t feasible. This carousel of video formats, or codecs, never stops. So what is a poor content factory to do? That’s why we invited Hiscale and Sarah Semlear to join us to talk about the bit-crunching processing engine of the content factory.

So I’m going to introduce our guest from Hiscale, Sarah Semlear. Sarah is a video expert who has worked at big-name brands such as Netflix, Tubescience, BuzzFeed and Comedy Central, and these days they run go-to-market for Hiscale, which, as Jeff said, makes an elastic transcoding solution that tells us that transcoding is boring. They’ve sat on both sides of this. They’ve been the person responsible for making the operation work, and now they’re on the vendor side helping other people do it. So welcome, Sarah.

Before we get into the products, Sarah, give people the frame in your own words. Hiscale says transcoding should be boring. What do you mean by that, and why is this the right goal?

Why Transcoding Should Be Boring (2:50)

Sarah: A lot of people think of transcoding last, right? You’re not coming into a scenario thinking, okay, I need to get my storage, I need to get my MAM, I need to get all these things set up. Oh, I also need transcoding. You’re thinking of that as something that’s baked into the system already. FFmpeg exists already, Vidispine exists already, behind the system that you’re installing. But you’re not necessarily thinking of those edge cases, or of the high volume that you’re going to have to churn through. And so it comes to us last. We don’t want to think about it. We don’t want to spend that time trying to get this thing that’s so fundamental set up.

So, transcoding is boring: yes and no. It’s a fundamental part of it, but it’s the math equation behind the system. And a lot of times when your systems are failing, let’s take Iconik for example, they’re a close partner of ours. You have your backlog, your whole big library that you’re trying to get in, years and years, terabytes and terabytes. You’ve never had a MAM before. You’re trying to load all that in. FFmpeg is going to burn out really fast, and you’re going to blame your MAM. It’s not your MAM’s fault. It’s your transcoder. So you should have to think about that less. Getting all that historical data in, and then churning through your daily, should be something you don’t have to think about. It should just work for you.

It’s not your MAM’s fault. It’s your transcoder.

Sarah Semlear

Two Ways In: Camera Cards and Hot Folders (4:16)

Jeff: Start at the front door. In our initial episodes, Eivind and I argued that corporate teams really underestimate everything that happens before anyone points a camera, and everything about getting content in cleanly. Hiscale has two ingest paths: the Hiscale AGENT and FLICS INGEST for live signals. Can you draw the map for us? What are those two, and when does a corporate team need each of those paths?

Sarah: Some teams need to ingest from camera. They’re on set, they’re coming back from set, they need to grab those camera cards and pull them in. You need to do certain functions on those incoming files. You need to move them to certain places. You want them ingested automatically into your MAM, you want them on your storage as well, you want them transformed into different formats so that they’re usable automatically. Maybe you want a LUT put on so that you can start editing right away. And you want them made into edit proxies and view proxies, for example. With the AGENT, we can easily pull that media in from camera cards, from hard drives, from other storage sources, so that we can get it to the right places in the format that we need.

If someone has something in their storage already and just needs it transformed, you can put it into a hot folder, and it ends up in your asset management system or transformed into different types of proxies and formats. The reason those two things, AGENT ingest and hot folders, are super important is that you can give them to non-technical humans. Super technical humans on the camera side, but not necessarily on the post-production side. We in this world know how different those things are. Maybe this person is an amazing DP, but they’re not the most technical when it comes to computers. You want to give them one thing to do. This is the action that can be performed, and it’s automatically going to transform your footage into the correct places and the correct formats. A one-stop shop from your raw files to something that’s editable, taking the place of some of those legacy transcoding systems.

As the content is pulled in, it’s transformed on the back end into how the asset management system presents that content. So it gives you a view proxy. It can also create edit proxies. It can also create share links. We can do really complex workflows. This is getting more into the realm where you have a media manager, you’re doing a lot more complex stuff, you need to send out things like dailies. So you need something in an asset management system where you can send that share file off. It depends on the specifics of the people involved, how technical they are, and what you need to happen to those files. Again, transcoding is boring. Think about it less. Give the right tools to the right person and you’ll get it done quicker and more effectively.

Give the right tools to the right person and you’ll get it done quicker and more effectively.

Sarah Semlear

Verified Ingest and a Free Agent (7:22)

Jeff: The AGENT app is free, which is kind of unusual. Love free stuff. It gives you verified transfers with checksums, pause and resume, metadata-based naming, and it can trigger downstream workflows. So why give that away, and what does a verified ingest actually protect a corporate team from?

Sarah: When we say it’s free, that’s from a camera card ingest point. The fancy copy-paste, where you’re moving files from your camera cards or your hard drive onto your file storage, or into your asset management system, or into AWS. That transfer function is free. We don’t charge for the download of the product or for that. I call it fancy copy-paste because it’s moving that file but also performing a checksum, making sure the file moved correctly. That’s really important, because files in movement can get corrupted, and you want to know if something was corrupt before it’s moved, so you don’t wipe your camera card and then have corrupted files.

We also do verified uploads to things like LucidLink and Suite. That’s super important because sometimes you think it transferred, but it only transferred into the cache. You shut your laptop, the transfer fails. This shows you when it has actually uploaded to LucidLink or Suite, to the cloud, not just to the cache but all the way into that storage.

But the real power in the tool comes from adding the transcode capabilities, and that’s not free. You pay for the nodes, you pay for concurrent transcodes, and you pay for the workflow aspect. The workflow automation is the really cool part, where you can ask the system to do all sorts of things with that file and to that file, so that it’s properly set up for your editor and for your producers. Let’s say you want the entire camera card stitched together into one file and then sent out to view as a daily. That’s something we can easily do. So that’s the paid portion. It is free for sure, but the cool stuff comes when you have the FLICS subscription.

Editing While It Records: Growing Files (9:54)

Jeff: That makes sense, because the one thing that really messed a lot of people up in broadcast was when segmented files came out, the chunks. People did not understand that. One thing I think is a really sharp capability is growing files. It’s not just for sports, it’s for corporate too: starting to edit and clip while the recording is still going on at the other end. Can you explain that, and where it changes the game for, say, a big internal live event?

Sarah: There are a couple of different ways we work with growing files. The first is on your active storage: your LucidLink, your Suite, your StorNext SAN, any active storage you have. As long as we can see that storage, we can create a growing file that you can pull into your editing system and start working on immediately. That’s an incredibly important workflow for a lot of live and sports. We have to be able to start clipping that content immediately.

But the really cool thing is that we can also create growing files in asset management systems. A good example is Iconik and Cantemo. I’ll go over Iconik briefly because, like I said, we have a close partnership with them and work really closely with them. As the file is coming in live, we can see it and play it back within Iconik. We can also create subclips that let us start working on those files as they’re growing, which is really cool. That’s a lot newer. We were only able to do that starting at the last NAB, and it’s pretty unique to us. We were the first to figure out how to create that growing file as something you’re able to play back. You can create growing files in asset management systems other than that, but without us you can’t play it back to see it, which is really important for producers. They’re not necessarily going to want to look at that on the active storage or in Premiere. They’re going to want to look at it in a scenario that’s easy for them to understand and digest.

Enterprise Video Transcoding: Elastic, Hybrid and Cost-Aware (12:16)

Jeff: I would say this is the front door of the factory done right. The content comes in verified, it’s structured, and it’s already moving, instead of arriving as a pile of unlabeled cards and discs that someone has to sort out later. Perhaps a traveling hard disk, like…

[laughter]

…we’ve talked about a few times. So let’s switch over to the loading dock, the other end. FLICS is an elastic transcoding tool for file, VOD and live. For a corporate audience that does not think about transcoding at all, what is elastic transcoding, why does it matter, and what goes wrong when it’s done badly?

This is the front door of the factory done right.

Jeff Sengpiehl

Sarah: The main thing that we wanted to make sure the elastic ability was there for was cost in the cloud.

Eivind: Okay.

Sarah: Cost is something we’re always focusing on, especially cloud cost. If you’re running an active environment in the cloud, it’s costing you whatever you’re using. The point of our transcoding system is that it’s able to spin up and down based on need. So it’s not costing you additional in the cloud, it’s not costing you something you shouldn’t have to pay for because it’s sitting there not doing anything. It spins up when there’s a job coming in and spins right back down, so you’re not paying the extra cost when your system isn’t active. That elastic ability is really important to reduce cost.

But let’s say you have an on-prem environment as well. That on-prem environment you don’t have to spin up and down. It’s stable, it’s always there, you have X amount of things coming through at all times. But because we’re super flexible in the way we’re set up, you can also have a cloud environment. So that flexibility comes back in: every month, or twice a year, I have these huge projects coming in. I need to be able to scale way up and pull in all this additional content so I can process it quickly and not bog down the system that’s churning through my regular day-to-day stuff. Or you have a couple of petabytes you want to get in from your archive as a one-time thing. You can process that through the cloud instead of bogging up the normal system.

And it works the same way if your normal system is set up in the cloud. We can also have spikes of consumable hours added on, so we can process everything quickly, not bog down your normal nodes, and get everything in sooner rather than later. There are a lot of important use cases, especially as we grow further into the cloud. It’s a huge benefit to be able to say, yes, we can scale up and down, and it’s native inside the platform.

Jeff: I think that also points to this: if you’ve got something urgent that comes through and it needs to go faster than all the rest of the normal content, it can pick the fastest method to deliver that content where you need it. That’s something not a lot of other folks are able to do.

Eivind: If we get a chance to look at that later on, Sarah, I would love to see how you monitor costs and the environment.

Sarah: Yeah, and we monitor all that for you. It’s in our dashboard, where we can look at exactly how much cost is associated with everything. But part of the point is that you don’t have to choose which thing to transcode first. You can do both, because there doesn’t have to be a priority. We can scale up as much as you need.

Hybrid Done Right and Broadcast-Grade Formats (15:58)

Eivind: I think this is a great example of something we’ve talked about in other episodes, and I’ve talked about at other venues: on-premises versus cloud versus hybrid. This is an example of hybrid done right, where the environment can flow elastically and dynamically back and forth based on actual demand.

Your format support covers broadcast grade, RAW, ProRes, DNxHD, UHD, HDR and adaptive bitrates, and you’ve got a UHD acceleration trick and split and stitch. A corporate team might ask why they need broadcast-grade output at all. What’s the answer for those folks?

This is an example of hybrid done right, where the environment can flow elastically and dynamically back and forth based on actual demand.

Eivind Sandstrand

Sarah: Sometimes you do, sometimes you don’t. It’s highly dependent on your workflow. I think a lot of folks don’t realize what broadcast quality is, or that they’re already using it. There are all these formats that aren’t supported by other systems. If you want that flexibility, and you want to use any camera person who comes along, you’re saying, oh my gosh, we have to use that person because they’re amazing, and they only use this camera. I’ve run into that a lot of times [laughter], when I’ve had to tell my producers, hey, we can’t actually process that video. With us, we can process anything. We can handle any files you throw at us.

We have a few codecs we don’t currently support, but that’s not a lack of capability. It’s just because no one is using them or has asked for them yet. In some cases they’re super expensive. The only one that doesn’t come with the subscription in the base price is Sony RAW, because it’s super expensive. Every other codec you’d be processing through our system is included in the base subscription price. So you’re not nickel-and-dimed. You can process whatever you want, whenever you want.

Sarah: And you’re not having to think about, oh wait, does my transcoder process this? That’s another part of transcoding is boring, think about it less. You don’t have to worry about what you’re putting into the system and what you’re going to get out not working.

Codec Licensing and Where Orchestration Fits (18:15)

Eivind: This is one of my pet peeves about some of these, shall we say, cheaper transcoders out there, without mentioning names. Some of them are open-source projects, which means that, A, they’re not officially supported by anyone, really, and B, in many cases they do not include the license to encode something in a particular format. This is perhaps not well known outside the broadcast world: some of these formats are licensed, and if you produce more than a certain amount of content, you’re supposed to pay for them. When all of this is fully sanctioned inside the application, that gives you the guarantee, the peace of mind, that you can produce in any format you want without getting bitten in the behind by it.

There’s an orchestration layer inside FLICS as well, with visual workflows, automation rules, and integration with MAMs, CMSs and cloud storage. We spent a lot of time in a previous episode talking about Embrace’s orchestration tools, specifically Pulse-IT, and Embrace is also one of your partners. How should people think about FLICS orchestration versus a dedicated orchestration layer, and how do they fit together?

Sarah: We work very closely with orchestration systems, and not just those but also the MAMs themselves. I think part of the question is, well, if you can do this, why would I have a MAM? There are a lot of different layers to these things. It’s hard, when there are so many options and systems out there, to understand how to move through it properly. My answer is that it depends on what you’re trying to do, unfortunately. It’s dependent on what the system needs to do.

So the first thing you look at is what you’re trying to do with the file and what systems you’re trying to move it between. If it has nothing to do with transcoding, it really shouldn’t be us doing the workflow for that file. Where we come in and shine is where the file needs to be transformed or edited in some way. A good example is the camera card coming in. I’m not just moving those files. I have to do something to them to make it easier for others to see them or start working on them. I’m transforming those files. Another good example where our workflows work really well is localization. You need a hundred different outputs of the same file with different end cards and different captioning based on the country. We can do that automatically. You push one button, and it’s still transforming the file. So we’re best involved when it has to do with transforming the file.

We don’t have a way to share files out securely. That’s a good example of where a MAM comes in. And when you just want a file to arrive somewhere and then go somewhere else and have a different system pick it up, there’s a lot of complexity, and an orchestration layer on top becomes incredibly useful. When you put it all into your transcoder, there are a lot [laughter] of points of failure. An orchestration system on top gives you that layer of stability and can talk to all the various pieces so you’re moving through it in the right order.

I think a lot of people come to conclusions where they want the simplest version of things. They want one system to rule them all. We’ve found that piecing things together is a lot more effective, because you have these tools that are so specialized and so amazing at what they do that together they make this really freaking cool thing. It’s stronger as an amalgamation than as individual pieces. We’re a really good transcoder. We’re really good workflow orchestration at the transcode and file transformation level. But that doesn’t mean we should be moving files all over the place, or trying to take the place of what an asset management system can do, and the power that comes with some of those automations, like making sure files are not getting lost, because that’s a huge problem in a lot of corporations.

Specialized Parts That Connect (23:13)

Eivind: This is a really key moment for our listeners. Two of our guests, Embrace and Hiscale, and others, are close partners, and this is the content factory idea working as intended. Specialized parts that connect, rather than having to be joined through a science project. They just connect, because they’ve been built like that. Rather than one box claiming to do everything and not doing it optimally.

Specialized parts that connect, rather than having to be joined through a science project.

Eivind Sandstrand

Sarah: Well, it used to be that people claimed they could be that all-in-one. As we grew as an industry and everything became way more complicated, they realized it’s actually better if we talk to each other and work together, and we’re friends.

Eivind: Yeah.

Sarah: And we can build something even cooler that way.

Eivind: It makes it a lot easier. And there’s a lot of feature development that happens when you do that. One company pushes another to do things they weren’t thinking about, and that’s where everybody ends up moving forward in terms of their workflow.

A favorite quote of mine is from Helge Høibraaten, CEO of CuttingRoom, who once said that innovation in isolation is damn near impossible. If you can’t partner and collaborate with other people, you’re stuck with your own ideas. They may be good for a while, but inspiration from outside is great. So I welcome and applaud the willingness to partner.

Sarah: That’s not to say we can’t come up with systems that are pre-baked for folks, if that makes sense. We work really closely with Embrace and Iconik and Suite and LucidLink. We already have integrations with everyone, pre-baked for you. You don’t have to think about it. So we can present these as a full package, versus something disparate, where you go talk to them separately: I actually know the salesperson, you can go talk to them over there.

Part Two Is Coming (25:21)

Jeff: Sarah, thanks for joining us. Next time we’re going to dig into another part of the content factory. And if you’re a vendor who’s up for cutting through the BS, drop us a line. Thanks, everybody, for joining us today. Next time we’re going to dig into the demo of what Hiscale does, with Sarah. Please make sure you join us for part two. As always, hit those like and subscribe buttons. It helps us bring this content to you and get it to the right audience. Our contact information is below, so you can get hold of either Eivind or myself to discuss anything that is bothering you in terms of your needs for a video content factory. We look forward to seeing you on our next episode. Thanks, everyone.