The media company is headquartered in Kolkatta. had a massive inventory of data in TB. Managing and migrating such large streams of rich content required for them to use huge volumes of manpower expertised in the implementation of cloud migration.
The company continuously uploads new digital content on its platform and anticipates seamless media consumption for its target audience. The tape drive backup system, because of its portability and adaptability limitations made it relatively challenging to provide its audience with instant and endless access to content-rich programs. The company wanted to achieve cost-effective outcome while retaining their expectations in performance, stability, compatibility, and compliance
Minfy extracted 100 TB data out of the Tape Drives and moved on to AWS using AWS Snowball. Files transferred using Snowball were also moved directly to Glacier for archival. AWS File Gateway was configured to move daily delta files directly on to AWS S3 Files stored on AWS S3 as the primary storage with Intelligent Tier enabled. The Intelligent storage class is a low cost and optimizing it by the data access patterns, not compromising with the performance or operational overhead. It moves data between two access tiers – frequent and infrequent access tiers – when access pattern changes, ideal for data with changing access patterns.
The data is moved to Glacier from Amazon S3 based on lifecycle policies via the AWS Management Console, which set rules to configure the movement of data to Glacier. It can be either transition or expiration which decides the cost of data management.
This is applied to the S3 buckets, by selecting the appropriate bucket and going to the Management tab to Add Lifecycle Rule. Here the rules are added with a name and selecting what versions of data are transferred, after how many days. Later an expiration date can be specified, which will permanently delete data from S3 after a given number of days. The files can be restored from Glacier to be moved to S3 by specifying how long to be accessible on S3 and retrieval types – Bulk, Standard and Expedited.
The lifecycle rules are highly useful when the data only needs to be available on S3 for a specified amount of time like periodic logs or some those need to be archived like digital media as in the case of our project. Broadly there are GET bucket lifecycle, PUT bucket lifecycle and DELETE bucket lifecycle.