Design File Storage and Sync
A Dropbox or Google Drive case study covering chunking, deduplication, a metadata database, block storage on S3, sync notifications, conflicts and versioning.
Requirements
A file storage and sync service keeps your files in the cloud and mirrors them across your devices. Edit a file on your laptop and it appears on your phone. Dropbox, Google Drive and OneDrive work this way.
Functional:
- Upload, download and delete files and folders from web, desktop and mobile.
- Sync: a change on one device reaches the user's other devices automatically.
- Share files and folders with other users.
- Version history: view and restore earlier versions.
Non-functional:
- Reliable: never lose a file. This is the most important requirement.
- Efficient: editing a small part of a large file must not re-upload the whole file.
- Fast sync: other devices learn about a change within seconds.
- Consistent metadata: a user must not see a half-uploaded file or two different folder trees.
1 of 10