technology – Clustering For Mere Mortals

I’ve had a Dropbox account for years. Like a lot of people, I started using it when it was one of the easiest ways to sync files across machines and keep important stuff backed up.

Over time, that Dropbox account became a bit of a digital attic. Old recordings, videos from various bands over the past 12 years, charts, rehearsal tracks, PDFs, installers, and all kinds of things I didn’t necessarily need every day, but definitely didn’t want to lose.

The problem was that I was paying about $20/month to keep it around.

That might not sound like a lot, but it adds up to about $240/year. And since I already pay for Google storage and iCloud storage, it seemed silly to keep paying for Dropbox just because I had never gotten around to cleaning it up.

So the question became:

How do I move almost a TB of data from Dropbox to Google Drive without downloading it all locally and uploading it again?

Because honestly, who has the time or local disk space for that?

The Problem

My Dropbox account had roughly 734 GB of data in it.

My Mac only had about 30 GB of free space.

So the normal approach was not going to work.

I could not just sync everything locally from Dropbox and then drag it into Google Drive. There simply wasn’t enough room on the Mac.

I looked at a few paid cloud migration services. They would have worked, but the plan I needed was going to cost more than I wanted to spend for a one-time migration.

After a quick conversation with ChatGPT, I decided to use rclone.

rclone is a free command-line tool that can copy data between cloud providers. In my case, I used it to copy files directly from Dropbox to Google Drive.

The best part?

The data never had to live on my Mac.

My Mac simply coordinated the transfer.

Dropbox → Google Drive

No giant local download. No external hard drive. No paid migration service.

Step 1: Install rclone on macOS

I already had Homebrew installed, so installing rclone was simple.

Open Terminal and run:

brew install rclone

Verify the install:

rclone version

You should see version information similar to:

rclone v1.74.3

Step 2: Configure the Dropbox Remote

Run:

rclone config

Choose:

for a new remote.

Name it:

dropbox

When prompted for the storage type, choose Dropbox. In my version of rclone, Dropbox was option 15.

			
15 / Dropbox
   \ (dropbox)

For the client ID and client secret, I just pressed Enter to accept the defaults.

When prompted to use automatic configuration, choose:

A browser window opened and asked me to authorize Dropbox. Once I approved it, the browser showed:

			
Success!
All done. Please go back to rclone.

Back in Terminal, I saved the remote.

Step 3: Configure the Google Drive Remote

Run the config again:

rclone config

Choose:

Name this remote:

gdrive

When prompted for storage type, choose Google Drive. In my version of rclone, it was option 24.

			
24 / Google Drive
   \ (drive)

Again, I left the client ID and client secret blank by pressing Enter.

For scope, I selected full access:

For the service account file, I pressed Enter.

When prompted for advanced configuration:

When prompted to use the browser for authentication:

This opened a Google authentication window.

This part is important. I made sure to authenticate with the Google account I actually wanted to use as the destination:

dave@medfordband.com

That is my personal Google Workspace account, and it already had enough storage available.

After approving the permissions, I saved the remote.

Step 4: Verify Both Remotes

To confirm both remotes were configured, I ran:

rclone listremotes

The output showed:

			
dropbox:
gdrive:

Then I verified that I could see files on both sides.

rclone lsd dropbox:

and:

rclone lsd gdrive:

Dropbox showed my old folders. Google Drive showed my existing Google Drive folders.

At this point, I knew the connections were working.

Step 5: Run a Small Test Copy

Before moving hundreds of gigabytes, I tested with a small folder.

rclone copy "dropbox:Scripts" "gdrive:Dropbox Migration/Scripts" --progress

That completed successfully.

The output showed that 64 files had transferred.

Good enough for me.

Step 6: Keep the Mac Awake

Since this was going to run for hours, I opened a second Terminal window and ran:

caffeinate -d

That command does not return any fancy output. It just sits there.

That is normal.

It keeps the Mac awake while the transfer runs. Leave that Terminal window open until the migration finishes.

Step 7: Start the Full Dropbox to Google Drive Migration

In the original Terminal window, I started the full copy:

			
rclone copy dropbox: "gdrive:Dropbox Migration" --progress --transfers 8 --checkers 16

That command copies everything from the root of Dropbox into a folder in Google Drive called:

Dropbox Migration

The --progress option shows live progress.

The --transfers 8 option allows multiple files to transfer at once.

The --checkers 16 option lets rclone check files more aggressively while copying.

Once it started, I could see files actively moving.

			
Transferred: 252.220 MiB / 449.554 GiB
ETA 5h23m54s

At that point, I knew the migration was off and running.

What If It Gets Interrupted?

One nice thing about rclone is that it is resumable.

If the Mac reboots, the network drops, or the process gets interrupted, just run the same command again:

			
rclone copy dropbox: "gdrive:Dropbox Migration" --progress --transfers 8 --checkers 16

rclone will skip files that already copied successfully and continue with the rest.

Step 8: Verify the Migration

After the copy finishes, compare the source and destination sizes.

Check Dropbox:

rclone size dropbox:

Check Google Drive:

rclone size "gdrive:Dropbox Migration"

The numbers should be reasonably close.

There may be some differences depending on shared folders, shortcuts, Dropbox backups, Google Photos references, or files that are counted differently by each provider.

Why I Like This Approach

This solved a very practical problem.

I had a lot of history in Dropbox that I did not want to lose. Old band recordings. Videos. Rehearsals. Charts. Random files collected over more than a decade.

But I was tired of spending $20/month just to keep that old account alive.

Since I already pay for Google storage, consolidating everything into my Google account makes sense.

The other options were not great:

Download everything locally? Not enough disk space.

Upload it all again manually? No thanks.

Pay for a migration service? Maybe, but I only needed this once.

With rclone, I was able to do the migration for free, without needing a 1 TB local disk, and without moving the data through my Mac’s storage.

Final Thoughts

This is one of those little cleanup projects I should have done years ago.

Dropbox served its purpose, but now I’ll save about $20/month and have my old data consolidated in Google Drive where I already have plenty of storage.

If you have an old cloud storage account hanging around just because you’re afraid to lose what’s in it, rclone is worth a look.

It is not as fancy as a commercial migration tool, but it gets the job done.

And sometimes that is exactly what you need.

The Necessity of Pre-Production Testing

One of the cardinal rules of patch management is to test updates in a pre-production environment before deploying them to production systems. This approach allows IT teams to identify potential issues and conflicts that may arise from the update, ensuring that the actual deployment does not disrupt business operations. By simulating the production environment, organizations can validate the stability and compatibility of the updates, significantly reducing the risk of unforeseen failures.

Fast-Tracking Security Updates with Caution

While regular updates should follow a structured testing process, security updates that address zero-day exploits present a different challenge. These updates often need to be fast-tracked to mitigate the immediate risk posed by the vulnerabilities they address. However, this rapid deployment must be balanced with strategies to minimize associated risks.

One effective strategy is clustering business-critical applications and applying rolling updates. In a clustered environment, updates can be applied to just the active node, allowing it to run and validate the update’s stability. Only after the update has proven stable should the update be applied to the backup node. This staggered approach ensures that there is always a functional node available, significantly reducing the risk of total system downtime.

Collaboration with SaaS Providers

The recent outage also highlights the crucial role that SaaS providers play in patch management. Companies like Palo Alto Networks and CrowdStrike, whose services are integral to their customers’ security postures, must collaborate closely with their clients to ensure updates are managed effectively. This collaboration should include clear communication about the nature and importance of updates, as well as providing tools and guidance to facilitate smooth deployment.

For instance, Palo Alto Networks recently faced an issue with a firewall bug that caused widespread disruptions. Such incidents underscore the need for robust patch management practices and proactive communication between SaaS providers and their customers. By working together, they can develop strategies to deploy updates with minimal impact on operations, such as phased rollouts and extended support for older versions during critical periods.

Conclusion

The recent Microsoft outage serves as a poignant reminder of the importance of effective patch management. By testing updates in pre-production environments, fast-tracking security updates with caution, and fostering collaboration between SaaS providers and their customers, organizations can mitigate the risks associated with software updates. These best practices not only enhance system reliability but also ensure that business operations remain uninterrupted, safeguarding the trust and satisfaction of clients and stakeholders.

Effective patch management is not just a technical necessity; it is a critical component of maintaining business resilience in an increasingly digital world.