Cluster migration procedure

Who does this article apply to?

This article only applies to customers who have already purchased a DreamObjects storage plan.

You can check if you currently have a DreamObjects plan in your panel on the (Panel > 'Cloud Services' > 'DreamObjects') page. If you do not see a current plan already set up, you can ignore this article.

Overview of changes

Old DreamObjects accounts were configured on a cluster located on the West coast of the United States.

Beginning June 21, 2018, DreamHost began allowing users to migrate their existing DreamObjects accounts to a new and improved cluster on the East coast of the United States. This new cluster will help with the long term stability of your data.

What cluster is my DreamObjects data currently on?

You can view which cluster your data is currently stored on in the panel. Navigate to the  (Panel > 'Cloud Services' > 'DreamObjects') page. 

dho migration

Next to your DreamObjects username, look for the hostname. It will either point to the old WEST cluster, or the new EAST cluster. The example above shows the user is on the new EAST cluster.

What hostname do I use to connect to my DreamObjects data?

If you're on the new EAST cluster you would use the hostname: objects-us-east-1.dream.io

If you're on the old WEST cluster you would use the hostname: objects-us-west-1.dream.io

What will happen to my current data on the old West coast cluster?

Your data will continue to exist and function normally until October 1, 2018, after which the old cluster will be shut down.

Due to several technical incompatibilities, it's not possible to automatically migrate your data to the new cluster. However, please note that you have until October to migrate your data to the new cluster.

The 'Migration tool' described below contains step-by-step instructions on migrating your data using the DreamHost panel.

You'll still be able to add data to existing buckets (on the old West cluster) until October 1st.

However, you will not be able to add any new users or buckets to the old West cluster. Any new users or buckets you create must be on the new East cluster.

What will happen if I'm still using a Legacy CDN name?

DreamObjects CDN names are no longer supported at DreamHost. If you are currently using a 'Legacy' CDN name that is still functional, you must use the new cluster's hostname objects-us-east-1.dream.io after you migrate your data to the new cluster.

You can also create a new subdomain alias.

Migration tool

You might be familiar with the earlier version of this tool from the article Migrating 3rd party S3 buckets to DreamObjects.

The migration interface has been improved to allow you to select a DreamObjects user as the source for a copy, as well as the destination for a copy. You do not need to specify the Access Key or Secret Key for the source user, as they are copied internally in the migration tool after a source user is selected.

Tool limitations

  • The destination bucket MUST NOT contain any files.
  • Bucket & Object ACLs:
    • This includes public-read/AllUsers, as well as AuthenticatedUsers
    • Single-owner ACLs are preserved
    • Support for Group ACLs and Complex ACLs is in progress, but not available at this time.
  • CORS:
    • The migration tool does not currently support the transfer of bucket-specific CORS policies. If you are using CORS on your bucket, please note any CORS setting you may have on buckets in objects-us-west-1 and then apply those setting to the migrated bucket on objects-us-east-1.
  • Static Website:
    • The migration tool does not currently support the transfer of of any static sites you may have set up in objects-us-west-1 to objects-us-east-1. Please reconfigure your buckets with static site(s) once the transfer is complete.
  • Server-Side Encryption (SSE)
    • AWS SSE is not officially supported, but SSE-C is available
    • If you use SSE-C, you will NOT be able to use the migration tool because it does not have access to your encryption key

Requirements

  • A source bucket with data
  • A destination bucket name (or an empty bucket)

Creating a new user on the new EAST cluster

The panel should have already duplicated your WEST cluster username as an EAST cluster username. For example:

The EAST cluster user already has the corresponding bucket set up, but nothing is in it. If you see the EAST user already, skip this section and continue with migrating your data.

If for reason you do not see a corresponding EAST cluster username, you can always create it by following these steps.

  1. To migrate data to the new East cluster, you must first create a user on that cluster. Navigate to the (Panel > 'Cloud Services' > 'DreamObjects') page.
    dho migration
  2. At the top, click the Create New User button.
    dho migration
  3. In the 'Cluster' dropdown menu, make sure to select 'objects-us-east-1.dream.io'.
  4. Add a new username. This can be the same name as your current user because it will be on a different cluster. However, it's up to you to name it anything you like.
    dho migration
  5. Click Add User. The panel will take a few minutes to create the new user. When it's finished, the new user will show '0 Buckets'.
  6. You can proceed with migrating your data to this new user.

Migrating buckets to the new East cluster

  1. Navigate to the (Panel > 'Cloud Services' > 'DreamObjects') page in your DreamHost panel.
    DreamObjects Migration
  2. Click the 'Migrations' tab.
    DreamObjects Migration
  3. Click the + Create Migration button.
    DreamObjects Migration
  4. Click the 'Type' dropdown and select the DreamObjects username you'd like to migrate.

    Make sure to choose a username on the WEST cluster since this is where you'll be migrating data from.

    DreamObjects Migration
  5. When you do this, the Host, Access Key, and Secret Key fields populate and become un-editable. Click the Choose Destination & Buckets button.
    DreamObjects Migration
  6. Name — Enter a name for your migration. The name is for your use only.
  7. Destination User — Choose which user you'd like to use for the migration. The dropdown separates users by cluster.

    Make sure to select a user under the 'objects-us-east-1.dream.io cluster.

  8. Click to select or deselect the buckets you wish to migrate, or use the Select All or Select None buttons.
  9. Click the Start Migration button.

    If you just created this new user on the East cluster, you should see no bucket naming conflicts since there are no buckets yet under this new user.

    However, if you've already added buckets to the East cluster user, there could potentially be a naming conflict.

    If any of the bucket names you have chosen to migrate conflict with a bucket name in the destination cluster, the Bucket Conflict page appears before the Migration can start.

    A bucket conflict means that the name of bucket from the source cluster already exists in the destination cluster, and that the bucket in the destination cluster is not empty (nothing will be overwritten).

    DreamObjects Migration

    New bucket names are generated for you. You can either accept them, or change the name to something new.

    Click the Resolve Conflicts button to continue.

    DreamObjects Migration
  10. You are returned to the 'Migrations' tab where your new migration shows as 'In Progress', or 'Queued'.
    DreamObjects Migration
  11. When a migration completes, the panel shows the status as 'Complete' and it is moved to the 'Completed Migrations' section. An email confirmation is also sent to the primary email address of your DreamHost account. 

Using the new hostname after the migration

Once you have migrated your data to a bucket on the new cluster, you can start using the new bucket immediately.

If you use URLs or hostnames to provide public/website access to the objects in your buckets, you must change them from the old hostname (objects-us-west-1.dream.io) to the new hostname, objects-us-east-1.dream.io.

If you use a subdomain alias on the old cluster, make sure to add the alias to the new bucket to keep using it as the hostname.

Did this article answer your questions?

Article last updated PST.