wagoe

The storage library provides file storage with local, S3 and Google Cloud Storage backends behind a unified protocol.

Basic operations

(require '[wagoe.storage.ports :as ports])

;; Store a file
(ports/store-file storage
  {:bytes (.getBytes "file content") :content-type "text/plain"}
  {:filename "document.txt"})
;=> {:key "ab/1234-uuid-hash.txt" :url "..." :size 7 :content-type "text/plain" :stored-at #inst"..."}

;; Retrieve a file
(ports/retrieve-file storage "ab/1234-uuid-hash.txt")
;=> {:bytes #bytes[...] :content-type "text/plain" :size 12}

;; Check existence
(ports/file-exists? storage "ab/1234-uuid-hash.txt")

;; Delete
(ports/delete-file storage "ab/1234-uuid-hash.txt")

;; Signed URL (local signs only with :signing-secret, else returns the public URL)
(ports/generate-signed-url storage "ab/1234-uuid-hash.txt" 3600) ; expires in 1 hour

File validation

(require '[wagoe.storage.core.validation :as validation])

;; Validate before storing
(validation/validate-file
  {:bytes file-bytes :content-type "image/png"}
  {:filename "photo.png"}
  {:max-size           (* 5 1024 1024)  ; 5 MB; default 10 MB
   :allowed-types      #{"image/jpeg" "image/png" "image/webp"}
   :allowed-extensions #{"jpg" "jpeg" "png" "webp"}})
;=> {:valid? true :data {...}} or {:valid? false :errors [...]}

Image processing

(require '[wagoe.storage.ports :as ports])

;; Resize image; give one dimension to keep the aspect ratio
(ports/resize-image image-processor file-bytes {:width 800})

;; Create thumbnail: the longest side becomes 150 px
(ports/create-thumbnail image-processor file-bytes 150)

Integrant configuration

;; Local filesystem (development); :provider defaults to :local
:wagoe/storage
{:provider       :local
 :root           "/var/app/uploads"          ; or :base-path
 :url-base       "http://localhost:3000/api/v1/storage"
 :signing-secret #env STORAGE_SIGNING_SECRET}

;; AWS S3 (production)
:wagoe/storage
{:provider   :s3
 :bucket     #env AWS_S3_BUCKET
 :region     #env AWS_REGION
 :access-key #env AWS_ACCESS_KEY_ID          ; omit both for the default
 :secret-key #env AWS_SECRET_ACCESS_KEY}     ; AWS credentials

Storage key format

Keys are generated:

  • Local: {shard}/{timestamp}-{uuid}-{hash16}.{ext} where shard = first 2 chars of the content’s SHA-256, or {path}/{filename} when the caller names a :path

  • S3: {prefix}/{timestamp}-{uuid8}.{ext}

The UUID makes every key unique, so storing the same file twice stores it twice.

See storage library for the full reference.