How to analyze and reduce S3 storage usage?

S3 is an object store, not a file system. Object storage comes with significant advantages: unlimited storage capacity, high availability, and durability. However, there are some disadvantages too. For example, it is cumbersome to calculate the storage usage of a specific prefix (also known as a folder) in S3.

Luckily, there is a better way to analyze S3 storage usage.

• S3 Inventory provides CSV, ORC, or Parquet files listing all the objects stored within an S3 bucket on a daily or weekly basis.
• Athena queries CSV, ORC, or Parquet files and analyzes data on-the-fly.

Full article