Online video streaming company — and heavy Amazon cloud user — Netflix has open sourced Janitor Monkey, a tool that finds unused cloud resources and deletes them, leading to potentially valuable cost savings for customers. The move might be an outcome of last week’s Christmas outage that Netflix users faced after AWS EC2 downtime.
There is a growing ecosystem of cloud cost management tools on the market that track customers’ usage of cloud resources, and specifically resources in Amazon’s cloud. But Janitor Monkey is a free and open source tool available for download from GitHub.
It’s from Netflix’s Simian Army, which is a family of programs made by Netflix engineers to optimize their usage of Amazon’s cloud. Last year, Netflix open sourced Chaos Monkey, which is a free way for Amazon customers to test fault tolerance by randomly shutting down instances to see if the system is architectured for continuity in the event of downtime.
Janitor Monkey works in a simple three-step process: mark, notify, delete. The system is set by default to run on weekends — but it can be highly customized — and scans running resources in Amazon’s cloud. It alerts the owner of resources via email when it finds a resource that it believes could be deleted.
For example, it scans virtual machine instances and Elastic Block Storage (EBS) volumes; if it finds an EBS volume that has not been attached to an instance in the past 30 days, it will recommend deleting the EBS Volume. Users can customize how long the volume must be inactive for Janitor Monkey to trigger an alert. By default, Janitor Monkey will send an email to whoever has been specified to receive it, recommending that unused instance be deleted. If the owner wants to keep the instance and not delete it, a flag can be created marking it as an exception. Or, the owner can delete the resource immediately, or wait two days for Janitor Monkey to delete it.
“Over the last year Janitor Monkey has deleted over 5,000 resources running in our production and test environments,” Netflix engineers wrote in a blog post announcing Janitor Monkey. “It has helped keep our costs down and has freed up engineering time which is no longer needed to manage unused resources.”
Janitor Monkey creates a log in AWS’s SimpleDB service tracking its work. Netflix says for most users, the table is small enough that the database fits within Amazon’s free usage tier. Netflix says it plans to continually develop Janitor Monkey as well, including broadening the scope of resources it monitors. Right now, Janitor Monkey scans instances, EBS Volumes, EBS Volume Snapshots and Auto Scaling Groups. Netflix hopes to include launch configurations, security protocols and Amazon Machine Images (AMIs). Netflix also has a user interface to control Janitor Monkey, which it plans to open source in the future as well.
There is a growing group of tools to help companies track and optimize their cloud usage. Some are specific to Amazon’s cloud, such as Newvem, which recently began charging money to use some of its services. Other players in the market include Cloudyn and CloudCruiser, among others. In addition to providing analysis of cloud usage, those services also recommend to customers which specific instances types are best sized and most cost-effective for the customer to use, which are features beyond the scope of the Janitor Monkey tool.
But anyhow, as a cloud provider, customers that build and share tools that work to improve certain facets of a service is great. It creates loyal customers, and helps providers understand how their customers use their services.
