Shared Storage Part 1: SAN, NAS, Bandwidth, and Connections

5 THINGS - Simplifying Film, TV, and Media Technology

Be sure to check out the complete series on shared storage – 3 episodes!

This 3 part series consists of:

Part 1: SANs, NASes, Bandwidth, and Connections.
Part 2: Drives: Size, Spindles & Protection (RAID)
Part 3: Management, Permissions & Support

What we won’t be covering is cloud, LTO or ODA storage. Simply spinning disk. However, I may cover these in a future episode. Let’s stick with local and shared storage for now. Now, some of you may know my affinity for good scotch and craft beer. In fact, a past episode had a case of a current favorite brew in the background. Thus, it was suggested that I do an episode while enjoying a libation – and so here we are. Today, I’m enjoying Laphroaig.

So, while I pour myself a dram, let’s get ready to make 3 backups with episode 1: SANs, NASes, Bandwidth, and Connections.

1. What is a SAN and a NAS?

Before we jump into the geeky goodness, let’s make sure we’re all following the same nomenclature.

SAN: Storage Area Network.

A SAN, or Storage Area Network, is centralized storage that multiple computer systems can connect and use. Historically SANs were fibre channel based, however, by using different protocols over Ethernet, a SAN doesn’t HAVE to be Fibre based. Many times SANs have a proprietary connection method to each client machine which optimizes the performance between the two. They typically appear as local volumes, as opposed to network volumes. Which leads up to our next term:

NAS: Network Attached Storage.

A NAS is also a centralized storage pool and has historically been based on Ethernet connectivity. A NAS usually uses built into the OS protocols for client computers to connect to the storage. This means it’s easier to connect and to use the storage, but often performance is decreased because of the fast and loose protocols.

As I mentioned, there are 2 predominantly used physical methods of connecting many computers to the same storage: Ethernet and Fibre Channel. These physical transmission methods of Ethernet and Fibre Channel carry data in various protocols over the physical cable. Fibre channel commonly uses the iSCSI protocol, while Ethernet commonly uses AFP, SMB, or CIF. iSCSI needs an initiator (the client computer) and the server (the target). These are usually developed by 3rd parties and are not stock on your computer. AFP, SMB, or CIF shares are Windows and Mac protocols that virtually every machine should understand. Another way to differentiate is if the shared storage shows up on your computer as a “network volume” or “local storage”.SAN volumes usually appear as local volumes where NAS solutions appear to the computer as network shares. Many shared storage solutions are mixing the protocols found on Fibre Channel SANs and utilizing Ethernet to transmit them. This where the blending of the 2 technologies and 2 transmission methods converge.

The endgame is that when both of these topologies (when configured and deployed correctly) can be run long distances, are somewhat flexible, and most importantly, can be used for utilizing mass storage in the video realm. The big difference to the end user is sustained bandwidth.

2. What is bandwidth and why is it important?

Bandwidth – also known as throughput – is the amount of constant data your computer can read and write to and from storage and is paramount for video recording and playback. Too little bandwidth – or poor sustained bandwidth – causes your video to stutter or even refuse to play. In addition, this latency can cause the responsiveness of your NLE to tank. This is why choosing the correct Fibre Channel or Ethernet solution is so important.

Let’s take a look at the bandwidth you’ll find via Ethernet and Fibre Channel.

Now, first thing you’ll notice is that real-world throughput differs from theoretical throughput (20%-30%)

Kind of like that bag of life-shortening potato chips you picked up at the grocery store. Is it ever really filled to the top? Or the Miles Per Gallon (MPG) you’re supposed to get in your car that you never quite reach. Real world performance vs mathematical performance. Never forget they are not identical and can vary from solution to solution. An assumption we have to make with these numbers is that the storage’s drives and chassis can actually HANDLE the throughput the transmission method can handle- but we’ll dive into that more in-depth in the next episode.

If the storage can handle it, I normally ballpark 20-30% off of theoretical max for a more real-world ballpark estimation. ALL shared storage solutions are a little bit different. However, for sake of discussion, and to be safe when planning your facility, let’s say 30%. You always want to plan for the worst when designing a workflow and plan for future expansion. In addition, all of these connections have the ability to be bonded, that is, the aggregate speed of the multiple connections yield more throughput than a single connection. This ability, of course, is based on the SAN or NAS, the switch, and the Fibre or Ethernet card in your system supporting it. Never assume any of them handle aggregation– verify. If they do, then you now have an opportunity to potentially almost double your theoretical bandwidth highway. This is a very good thing. Now that we know how wide our highway is, let’s see how many cars we can fit on it.

3. How much bandwidth does my video take up?

When it comes to bandwidth, the video industry is actually in an excellent place right now. Acquisition codecs, like h.264 (Canon 5D, 7D), XDCAM, P2, and to some extent, RED, take up a small amount of bandwidth, while still delivering decent quality. In fact, for most editorial needs, bandwidth needs to each system have not grown very much in the past decade or so. I showed this chart similar to this a few weeks back, but it’s just as applicable now. Let’s look at old Standard Definition (SD) video data rates for example.

Ten plus years ago, broadcast networks would play material to air that had been onlined at 1:1 uncompressed or frequently at 2:1. The video data rate of SD at 1:1 is 22 megabytes (MB) per second. The current broadcast quality HD Avid codec (DNxHD 145) is actually less, clocking in at about 18MB/s. Even DNxHD 220x – the least amount of compression before going uncompressed in the Avid realm- is only about 27 MB/s. Thus, we can take advantage of the drops in storage pricing, and the advancement of computing power and use only slightly more than same bandwidth we were using for Editorial back in the SD days.

4. How many streams of video can I play?

Now that we know our connection methods, how much data we can push and pull down each pipe, as well as the data rates of popular CODECs, now we can see how they all fit together to best plan out your network topology.

So, let’s now match up our cars (CODECs) against our highway (throughput). Below we see codecs vs bandwidth (remember, bandwidth calculated at less than theoretical max, which is a better approximation of real-world performance)

So, what does this mean? Clearly, most codecs can easily fit through the pipeline 1GigE provides.

Fibre Channel becomes the obvious solution when uncompressed HD or higher resolution, less compressed CODECs come into play. Thus, for most video-centric editorial applications, an Ethernet-connected solution – single or bonded (even better) – can handle the job. This, of course, takes into account your computer is fast enough to decode the streams….the act of getting the data to and from your computer is different than the ability of the editing system to decode and display it.

5. What else do I need to know about Bandwidth?

I would add that not all Ethernet solutions are created equal. Sustained throughput is the name of the game, and off-the-shelf or I.T. centric solutions almost never handle video properly. Basic Ethernet topologies are relatively inefficient, and it is very common to have peaks and valleys with the transmission of data. These valleys can cause dropped frames. Video-centric shared storage solutions tweak the communication between the client machines and the server to optimize the video editing experience and maintain editing responsiveness. I cannot stress enough to look into video shared storage solutions, as opposed to basic Windows or Mac networking protocols. Talk to a video professional.

A Pinto can get onto a race track, doesn’t mean it performs well, ya know?

In closing for the 1st entry in this 3 part series, (1 for 3 may be good in baseball, but not in Post, and certainly not when choosing a shared storage system) I cannot stress enough that if you’re considering a SAN or NAS, consult a Video Shared Storage specialist. While I pour myself another glass -Did I miss anything essential about SANs, NASes, Bandwidth, and Connections? Have a suggestion on good Reds, Stouts, Porters or Scotch? Let me know in the comments, via Twitter, or on Facebook, and please, share this series with your friends. Stay tuned for Part 2: Drives: Size, Spindles & Protection (RAID). Same bat time, same bat channel. As always, thanks for watching.

Be sure to check out the complete series on shared storage – 3 episodes!

This 3 part series consists of:

Part 1: SANs, NASes, Bandwidth, and Connections.
Part 2: Drives: Size, Spindles & Protection (RAID)
Part 3: Management, Permissions & Support