Data is an asset, especially for media platforms, streaming services, and social networks. When it comes time to move massive repositories of documents, images, or audio files between cloud storage providers, there are challenges to moving all of the data set at once.

Typical bulk migrations require moving all assets, including files that are years old and potentially never accessed again. This results in two major drains on resources: exorbitant egress charges from the source provider and excess storage costs for housing unused data.

Introducing Fastly’s On-Demand Migration

Fastly’s On-Demand Migration for Object Storage was designed to solve this exact problem for small objects, offering a flexible and cost-efficient alternative to bulk transfer.

On-Demand Migration takes a stateless, URL-driven approach to migration, fundamentally changing when and how data moves. Instead of moving data in a massive batch, including everything upfront, migration occurs on the first request for an object.

Here’s how it works:

A client requests an object from your Fastly Object Storage bucket.

If the object is not yet present, Fastly’s Object Storage service reads the asset from a publicly provided source URL (via the fastly-object-storage-source-url header).

The asset is fetched and simultaneously served to the requesting user and asynchronously written into your Fastly Object Storage bucket.

Crucially, the migration process utilizes Fastly’s network to affect the data copy, which provides the most** **efficient data migration by copying data to Fastly Object Storage directly from the provided source, eliminating the need for customer-managed compute resources and bandwidth.

This mechanism allows customers to migrate only their "working set," the subset of files actively being accessed by users, drastically reducing egress costs from the source provider and minimizing capacity required on Fastly Object Storage.

A streaming platform’s seamless shift

A leading streaming platform faced a critical challenge to "lazy load" images to a new application without waiting to migrate their whole library and handling new user uploads at the same time. Without Fastly’s on-demand migration, they would have to wait for a long, costly migration process to complete.

By implementing on-demand migration, they gained:

Cost Control: The streaming platform was able to avoid paying massive egress fees to move unused files, migrating only the images and audio files that users were actually requesting.

Minimal Disruption: The seamless transition enabled by on-demand migration allowed them to start shifting production traffic immediately.

Investment protection: They also utilized Fastly Bot Management to prevent automated scrapers from "pushing" the migration of assets before they were genuinely needed by end-users, ensuring maximum cost efficiency.

Beyond S3: Migration from any public source

The stateless nature of on-demand migration offers unparalleled flexibility. While it handles private S3 buckets using temporary presigned URLs, it also allows migration from any publicly accessible source. This means you can migrate content with any accessible URL.

Move smarter, save more

For businesses dealing with millions of assets, on-demand migration fundamentally changes the strategy behind cloud migration. It’s a key advantage for customers seeking to consolidate their edge delivery and storage while keeping costs in check. The success of the streaming platform proves that you no longer have to pay to move and store files you may never use.

Ready to migrate smarter? Start using Fastly Object Storage today or visit our documentation for more information.