What is Netflix Compute?
Our team at Netflix supports a general purpose Compute Platform that powers workloads (networked service applications, batch processing, and interactive services) from across the various aspects of the Netflix business. If something at Netflix uses a computer in the cloud, our team delivers services, support, and technologies to make that computer easier to use. While engineers using our offerings support their own workloads, we look at workloads “fleet wide” ensuring that all workloads are supported well.
Compute is part of a broader Netflix cloud infrastructure focused organization (Cloud Infrastructure Engineering aka CIE). CIE’s mission is to “increase fleet-wide agility, efficiency, and reliability of the Netflix cloud infrastructure, while reducing the operational burden on application owners”. Note that reliability includes aspects of resiliency, scalability, and security. Compute’s focus is well articulated by this mission with a small change:
Compute’s mission is to “increase fleet-wide agility, efficiency, and reliability of the Netflix cloud compute-related infrastructure, while reducing the operational burden on application owners”.
Team
Our team has deep expertise in the space of compute infrastructure. The team has experience building infrastructure not only at very large scale technology companies but also startups and enterprise industries. This background brings scalable operational approaches, balances innovation and the long term support needs of our technology, and allows the team to learn from each other. The team has a mixed set of backgrounds in life experience coming to Netflix from many places around the world.
Products
We support five products that power 10s of thousands of workloads and all Netflix engineers.
BaseOS — The entirety of the Ubuntu based operating system including user space and kernel. This technology powers both our Virtual Machines (via AMIs) and our container base layers (via Docker).
Bakery — Takes workload deployment artifacts and turns them into runnable workload images in a standard way that allows for best practices and common metadata to be applied to all workloads. Bakery is open source.
Registry — Hosts runnable workload images used by our runtimes when starting new instances. We built this on top of Docker Distribution adding features that scale better than the open source offering.
Titus — A container specific Compute platform hosting workloads at Netflix built on top of AWS EC2 Bare Metal. A Netflix specific IaaS integrating deeply with our other infrastructure, platform, and security systems. Titus is open source.
Compute Managed Batch (CMB)— Netflix has many purpose built batch processing frameworks (machine learning, personalization, content processing, data analysis, and more). CMB is the lowest, most common layer of these systems providing scalable, predictability, and efficiency scheduling.
