Flux7 DevOps Blog

Netflix Data Management & Platform Libraries Tools |Glossary Part 3

Feb 28, 2014 5:00:54 AM Flux7 Labs Developer Tools and Processes

This is Part 3 of our series on Netflix open source tools. You can find Part 1 here and Part 2 here.

Netflix In-Memory Data Management Tools

Netflix-Graph

Netflix handles billions of daily requests that require access to video metadata. Given the huge number of stored videos, data is stored on the traffic-handling server’s RAM to reduce latency. Data is packaged and shipped using two applications:

  • Video Metadata Platform (VMS) packages video’s synopses, titles, artwork and streams.

  • Netflix-Graph stores data as a directed graph of the relationship between entities like video characters and tags. It stores any type of data that can be “phrased” as directed graph.

Zeno

Zeno is a highly optimized framework that helps manage and update the giga-bytes of constantly changing data that are replicated in thousands of servers. It provides:

  • Duplicate dataset removal.

  • Efficient data deserialization.

  • Powerful debug tools.

  • Improved development-team agility providing a clear line of separation between a data model and operations.

  • The ability to maintain minimal changes required to keep data up to date.

Netflix Platform Libraries Tools

Archaius

Archaius is a configuration management library focused on dynamic properties from multiple configuration stores. It’s a Java library that provides APIs for dynamic access to properties. It provides:

  • Dynamic properties of specific values.

  • A polling framework for dynamic configuration sources.

  • Ready-made configuration sources.

  • Context management.

  • High throughput and thread-safe configurations.

  • Scala wrappers.

Denominator

Netflix open source tools

Denominator is a portable Java library for processing DNS clouds. It has pluggable backends for:

  • AWS Route 53

  • Neustar Ultra

  • DynECT

  • Rackspace Cloud DNS

  • Openstack Designate

Feign

Feign is a Java-to-HTTP client binder that allows binding with minimal code and overhead. It provides:

  • Creation of customized components.

  • Production of multiple interfaces.

  • Quick integration with other open source projects.

Karyon

Karyon is a framework and library that serves as a blueprint for implementing cloud-ready web services and that can clone fine-grained web services and applications. Karyon’s blueprint includes:

  • Bootstrapping, Libraries, and Lifecycle Management.

  • Runtime Insights and Diagnostics.

  • Pluggable Web Resources.

  • Cloud readiness.

Karyon is based on two main entities:

  • Component is a standalone entity that’s required for the application to work, but is not a direct dependency of the application.

  • Application is a class inside the service. There can only be one application for each service and all components are initialized before the application is initialized.

Ribbon

Ribbon is a library that supports inter-process communication in the cloud. Features include:

  • Pluggable load balancing rules.

  • Service discovery integration.

  • Built-in failure resiliency.

  • Load balancer integration with clients.

It currently has 3 sub-projects:

  • ribbon-core

  • ribbon-eureka

  • ribbon-httpclient

Servo

Servo is an application-monitoring library that generates monitors and metrics. A monitor has a configuration that is a set of key-value pairs. A metric is the value for a given monitor at a specific time.

There are three types of monitors:

  • Gauges are simple monitors that return current values, such as queue size and thread number. Servo gauge implementations include BasicGauge, MinGauge, MaxGauge and NumberGauge.

  • Counters come in three flavors: Basic, Resettable and PeakRate.

  • Timers indicate time taken for a particular event..

Blitz4j

Blitz4j is a logging framework built on top of log4j. It allows high-scalable logging while maintaining high performance. Features include:

  • Event monitoring.

  • Business intelligence reporting.

  • Debugging.

Blitz4j mitigates data loss and provides a mechanism to summarize data during log storms.

Governator

Governator is a library of extensions and utilities that extend Google Guice capabilities. Guice is a dependency injection container open sourced by Google.

Flux7 Labs

Written by Flux7 Labs

Flux7 is the only Sherpa on the DevOps journey that assesses, designs, and teaches while implementing a holistic solution for its enterprise customers, thus giving its clients the skills needed to manage and expand on the technology moving forward. Not a reseller or an MSP, Flux7 recommendations are 100% focused on customer requirements and creating the most efficient infrastructure possible that automates operations, streamlines and enhances development, and supports specific business goals.

Subscribe to Flux7's Blog

Posts by Topic

see all

Recent Posts