File storage is a core part of most applications, yet cloud based object storage has complicated file access from an application perspective, as although conceptually the application only cares about the files being stored, the actual implementation details can vary quite considerably and in many cases this impacts testing, deployment, etc.

This talk is largely about how to abstract the underlying details into an interface to greatly simplify file based storage at an application level. This is not only a useful abstraction, but also a great example of how interfaces work in the Go language. The solution ends up being elegantly simple and yet easily extendable.

Examples

The talk covers various file system implementations that all use a single interface:

Open(path string) (io.ReadCloser, error)

Example file systems discussed that all use this interface. Note: these can easily be extended.

• Local, Memory, URL, AWS S3, Google Cloud Storage
• Multiple cascading combinations
• Auto Retry
• Namespaced
• Prefixed
• Logging
• Traced
• Stat counting

Here are the slides for this talk. You can also check out the open sourced version of this library, feel free to add to it. Some of the examples are not fully completed, but basic code shells are listed with the talk.

‍