Cluster migration procedure

Beginning June 21, 2018, DreamHost will begin migrating existing DreamObjects accounts to a new and improved cluster. This will help with the long term stability of your data.

What will happen to my current data?

Your data will continue to exist and function normally until October 1, 2018, after which the old cluster will be shut down.

Due to several technical incompatibilities, it's not possible to automatically migrate your data to the new cluster. However, please note that you have until October to migrate your data to the new cluster.

The 'Migration tool' described below contains step-by-step instructions on migrating your data using the DreamHost panel.

You'll still be able to add data to existing buckets (on the old West cluster) until October 1st.

However, you will not be able to add any new users or buckets to the old West cluster. Any new users or buckets you create must be on the new East cluster.

What will happen if I'm still using a Legacy CDN name?

DreamObjects CDN names are no longer supported at DreamHost. If you are currently using a 'Legacy' CDN name that is still functional, you must use the new cluster's hostname objects-us-east-1.dream.io after you migrate your data to the new cluster.

You can also create a new subdomain alias.

Migration tool

You might be familiar with the earlier version of this tool from the article Migrating 3rd party S3 buckets to DreamObjects.

The migration interface has been improved to allow you to select a DreamObjects user as the source for a copy, as well as the destination for a copy. You do not need to specify the Access Key or Secret Key for the source user, as they are copied internally in the migration tool after a source user is selected.

Tool limitations

  • The destination bucket MUST NOT contain any files.
  • Bucket & Object ACLs:
    • This includes public-read/AllUsers, as well as AuthenticatedUsers
    • Single-owner ACLs are preserved
    • Support for Group ACLs and Complex ACLs is in progress, but not available at this time.
  • CORS:
    • The migration tool does not currently support the transfer of bucket-specific CORS policies. If you are using CORS on your bucket, please note any CORS setting you may have on buckets in objects-us-west-1 and then apply those setting to the migrated bucket on objects-us-east-1.
  • Static Website:
    • The migration tool does not currently support the transfer of of any static sites you may have set up in objects-us-west-1 to objects-us-east-1. Please reconfigure your buckets with static site(s) once the transfer is complete.
  • Server-Side Encryption (SSE)
    • AWS SSE is not officially supported, but SSE-C is available
    • If you use SSE-C, you will NOT be able to use the migration tool because it does not have access to your encryption key

Requirements

  • A source bucket with data
  • A destination bucket name (or an empty bucket)

Creating a new user on the new EAST cluster

  1. To migrate data to the new East cluster, you must first create a user on that cluster. Navigate to the (Panel > 'Cloud Services' > 'DreamObjects') page.
    dho migration
  2. At the top, click the Create New User button.
    dho migration
  3. In the 'Cluster' dropdown menu, make sure to select 'objects-us-east-1.dream.io'.
  4. Add a new username. This can be the same name as your current user because it will be on a different cluster. However, it's up to you to name it anything you like.
    dho migration
  5. Click Add User. The panel will take a few minutes to create the new user. When it's finished, the new user will show '0 Buckets'.
  6. You can proceed with migrating your data to this new user.

Migrating buckets to the new East cluster

  1. Navigate to the (Panel > 'Cloud Services' > 'DreamObjects') page in your DreamHost panel.
    DreamObjects Migration
  2. Click the 'Migrations' tab.
    DreamObjects Migration
  3. Click the + Create Migration button.
    DreamObjects Migration
  4. Click the 'Type' dropdown and select the DreamObjects username you'd like to migrate.

    Make sure to choose a username on the WEST cluster since this is where you'll be migrating data from.

    DreamObjects Migration
  5. When you do this, the Host, Access Key, and Secret Key fields populate and become un-editable. Click the Choose Destination & Buckets button.
    DreamObjects Migration
  6. Name — Enter a name for your migration. The name is for your use only.
  7. Destination User — Choose which user you'd like to use for the migration. The dropdown separates users by cluster.

    Make sure to select a user under the 'objects-us-east-1.dream.io cluster.

  8. Click to select or deselect the buckets you wish to migrate, or use the Select All or Select None buttons.
  9. Click the Start Migration button.

    If you just created this new user on the East cluster, you should see no bucket naming conflicts since there are no buckets yet under this new user.

    However, if you've already added buckets to the East cluster user, there could potentially be a naming conflict.

    If any of the bucket names you have chosen to migrate conflict with a bucket name in the destination cluster, the Bucket Conflict page appears before the Migration can start.

    A bucket conflict means that the name of bucket from the source cluster already exists in the destination cluster, and that the bucket in the destination cluster is not empty (nothing will be overwritten).

    DreamObjects Migration

    New bucket names are generated for you. You can either accept them, or change the name to something new.

    Click the Resolve Conflicts button to continue.

    DreamObjects Migration
  10. You are returned to the 'Migrations' tab where your new migration shows as 'In Progress', or 'Queued'.
    DreamObjects Migration
  11. When a migration completes, the panel shows the status as 'Complete' and it is moved to the 'Completed Migrations' section. An email confirmation is also sent to the primary email address of your DreamHost account. 

Using the new hostname after the migration

Once you have migrated your data to a bucket on the new cluster, you can start using the new bucket immediately.

If you use URLs or hostnames to provide public/website access to the objects in your buckets, you must change them from the old hostname (objects-us-west-1.dream.io) to the new hostname, objects-us-east-1.dream.io.

If you use a subdomain alias on the old cluster, make sure to add the alias to the new bucket to keep using it as the hostname.

Did this article answer your questions?

Article last updated PST.