For beginners

Upload files and images in Go to S3, R2 or MinIO

There are two ways to accept a file. The browser can send it to your API, which forwards it to storage, or your API can hand the browser a time-limited permission slip and let it upload directly. The second is almost always right, and the reason is not performance.

Both are below, with the checks that are easy to skip and are the ones that matter: a file upload is the most direct route a stranger has into your infrastructure.

17 min read. Commands and output last run against Grit v3.560.0.

What you will have at the end

Direct browser uploads to S3, Cloudflare R2, Backblaze B2 or MinIO, from Go.

Validation that works even though your server never sees the bytes.

Private files served through short-lived links rather than a public bucket.

Why not just POST it to your API

It works, and for small files it is fine. It stops being fine like this:

  • Your server holds the file while it uploads. A 200MB video over a slow connection occupies a request, a goroutine and memory or disk for several minutes. Twenty of those at once is your whole service.
  • Proxies have body limits. nginx defaults to 1MB and returns 413 before your handler runs. So does almost every load balancer and platform, each with a different number.
  • A redeploy kills the upload. Restart during a ten-minute upload and the user starts again.
  • You pay for the bandwidth twice, once inbound and once outbound to storage.

A presigned upload removes all four, because the bytes never touch your service. What it costs is that your server cannot inspect the file as it arrives, which is the thing the rest of this page is about.

What a presigned URL actually is

Not a secret link somebody generated. It is an ordinary storage URL with a signature in the query string, computed from your credentials, the exact operation, the exact object key and an expiry. Storage verifies the signature itself.

https://bucket.s3.eu-west-1.amazonaws.com/uploads/a1b2c3.jpg
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=AKIA.../20261121/eu-west-1/s3/aws4_request
&X-Amz-Date=20261121T143022Z
&X-Amz-Expires=300
&X-Amz-Signature=5d41402abc4b2a76b9719d911017c592...

Three consequences worth holding onto. The permission is for one operation on one key: a URL signed for PUT uploads/a1b2c3.jpg cannot write anywhere else. It expires, so a leaked URL is useless shortly afterwards. And your credentials are never sent to the browser, only a signature derived from them.

1. Browser asks your API
POST /uploads/presign
filename, size, type
2. API checks and signs
authorise, validate, generate a key
url + key
3. Browser PUTs the file to storage
your API is not involved
key
4. Browser tells your API it is done
POST /uploads/confirm

Step four is the one people leave out, and without it you have files in a bucket that nothing in your database knows about.

01Sign the URL

internal/services/storage.go
package services
import (
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"path/filepath"
"time"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
// Only what we are prepared to accept. An allow-list, not a block-list:
// a block-list is a list of the attacks somebody already thought of.
var allowedTypes = map[string]string{
"image/jpeg": ".jpg",
"image/png": ".png",
"image/webp": ".webp",
"application/pdf": ".pdf",
}
const maxUploadBytes = 10 << 20 // 10 MiB
func (s *StorageService) PresignUpload(
ctx context.Context, userID uint, filename, contentType string, size int64,
) (url, key string, err error) {
ext, ok := allowedTypes[contentType]
if !ok {
return "", "", ErrUnsupportedType
}
if size <= 0 || size > maxUploadBytes {
return "", "", ErrFileTooLarge
}
// Generate the key. NEVER build it from the filename the client sent:
// "../../etc/passwd" and "index.html" are both filenames, and the
// second one is how a bucket starts serving somebody else's page.
buf := make([]byte, 16)
if _, err := rand.Read(buf); err != nil {
return "", "", fmt.Errorf("generate key: %w", err)
}
key = fmt.Sprintf("uploads/%d/%s%s", userID, hex.EncodeToString(buf), ext)
req, err := s.presigner.PresignPutObject(ctx, &s3.PutObjectInput{
Bucket: &s.bucket,
Key: &key,
// Binding the content type into the signature means the upload is
// rejected if the browser sends something else.
ContentType: &contentType,
ContentLength: &size,
}, s3.WithPresignExpires(5*time.Minute))
if err != nil {
return "", "", fmt.Errorf("presign: %w", err)
}
return req.URL, key, nil
}

The original filename is still worth keeping, as a column, so the download can be called invoice-july.pdf. It is just never part of the path.

Everything the client tells you is a claim

Content-Type is set by the browser from the file extension, and by curl from whatever you type. size is a number in a JSON body. Checking them is worth doing, because it catches honest mistakes early, and it is not security. The real check happens after the bytes exist.

02Upload from the browser

async function upload(file: File) {
// 1. Ask for permission
const { data } = await api<{ data: { url: string; key: string } }>(
'/uploads/presign',
{
method: 'POST',
body: JSON.stringify({
filename: file.name,
content_type: file.type,
size: file.size,
}),
},
)
// 2. Send the file straight to storage. Note: no credentials, no cookies,
// and the Content-Type must match what was signed, exactly.
const put = await fetch(data.url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': file.type },
})
if (!put.ok) throw new Error('upload failed')
// 3. Tell the API the object exists
return api('/uploads/confirm', {
method: 'POST',
body: JSON.stringify({ key: data.key, filename: file.name }),
})
}

A progress bar needs XMLHttpRequest

fetch has no upload progress event. For a visible percentage, which is worth having for anything over a few megabytes, use XMLHttpRequest and its upload.onprogress. It is the one place the older API is still the right tool.

03Confirm, and verify what actually arrived

This is the step that makes the flow safe. Your server did not see the bytes, so it asks storage what it got.

internal/handlers/upload.go
func (h *UploadHandler) Confirm(c *gin.Context) {
var req ConfirmRequest
if err := c.ShouldBindJSON(&req); err != nil {
respond.ValidationError(c, err)
return
}
userID := c.GetUint("user_id")
// The key must be one WE issued to THIS user. Without this check, a
// caller can confirm any key they can guess, and attach somebody
// else's file to their own record.
if !strings.HasPrefix(req.Key, fmt.Sprintf("uploads/%d/", userID)) {
respond.Fail(c, http.StatusForbidden, "FORBIDDEN", "not your upload")
return
}
// Ask storage what is actually there: the real size and the real type,
// rather than what the client said five minutes ago.
head, err := h.storage.Head(c.Request.Context(), req.Key)
if err != nil {
respond.Fail(c, http.StatusBadRequest, "UPLOAD_MISSING", "no file at that key")
return
}
if head.ContentLength > maxUploadBytes {
_ = h.storage.Delete(c.Request.Context(), req.Key)
respond.Fail(c, http.StatusRequestEntityTooLarge, "TOO_LARGE", "file is too large")
return
}
upload, err := h.uploads.Record(c.Request.Context(), userID, req.Key,
req.Filename, head.ContentLength, head.ContentType)
if err != nil {
respond.Error(c, err)
return
}
respond.Created(c, upload)
}

Check the bytes, not the extension

A file called photo.jpg with Content-Type: image/jpeg can contain anything at all. The only honest check reads the first few bytes:

// The first 512 bytes are enough for http.DetectContentType, so download
// only those rather than the whole file.
head := make([]byte, 512)
if _, err := io.ReadFull(obj.Body, head); err != nil && err != io.ErrUnexpectedEOF {
return fmt.Errorf("read header: %w", err)
}
if !allowed[http.DetectContentType(head)] {
_ = s.Delete(ctx, key)
return ErrUnsupportedType
}

For images, decoding is a stronger check still, and you were going to decode anyway to make thumbnails. A file that is not a real image fails to decode, which answers the question completely.

Serving them back

A public bucket is right for avatars and product photos: they are cacheable, they are served by the CDN, and your service is not in the path.

It is wrong for anything a user uploaded that another user should not see. Private bucket plus a short-lived download URL:

func (h *UploadHandler) Download(c *gin.Context) {
upload, err := h.uploads.ByID(c.Request.Context(), c.Param("id"))
if err != nil {
respond.Error(c, err)
return
}
// The authorisation check happens HERE, in your code, which is the
// entire reason the bucket is private.
if upload.UserID != c.GetUint("user_id") {
// 404, not 403: a 403 confirms the file exists.
respond.Fail(c, http.StatusNotFound, "NOT_FOUND", "not found")
return
}
url, err := h.storage.PresignGet(c.Request.Context(), upload.Key, 5*time.Minute,
// Makes the browser save it under its original name rather than
// the random key.
"attachment; filename=\""+upload.Filename+"\"")
if err != nil {
respond.Error(c, err)
return
}
c.Redirect(http.StatusFound, url)
}

Guessable keys are not access control

A random key makes a URL hard to guess, which is worth something and is not a permission. Links get pasted into chat, forwarded, and indexed when somebody puts one on a public page. If the answer to "may this person see this file" is ever no, the bucket is private and your code decides.

Images need a second pass

Serving the 4000 by 3000 photograph somebody took on their phone as a 48-pixel avatar wastes their bandwidth and your money, on every page load.

// In a background job, not in the request: resizing is CPU work and
// nobody should wait for it.
func (w *ImageWorker) MakeRenditions(ctx context.Context, key string) error {
src, err := w.storage.Get(ctx, key)
if err != nil {
return fmt.Errorf("get original: %w", err)
}
defer src.Close()
img, _, err := image.Decode(src)
if err != nil {
// Not an image, whatever it was called. That is now known.
return fmt.Errorf("decode %s: %w", key, err)
}
for _, size := range []int{128, 512, 1280} {
resized := imaging.Fit(img, size, size, imaging.Lanczos)
var buf bytes.Buffer
if err := webp.Encode(&buf, resized, nil); err != nil {
return fmt.Errorf("encode %d: %w", size, err)
}
if err := w.storage.Put(ctx, renditionKey(key, size), &buf, "image/webp"); err != nil {
return fmt.Errorf("store %d: %w", size, err)
}
}
return nil
}

A stored file is several objects

Once there are renditions, deleting "the upload" means deleting the original and every derived file. Keep the list, on the record, rather than reconstructing it from a naming rule: the rule will change, and the old objects will stay in the bucket costing money forever.

S3, R2, B2 or MinIO

All four speak the S3 API, so the same SDK and the same code work against each. What differs is the bill and the endpoint.

  • AWS S3. The reference. Egress is charged per gigabyte and is what makes an image-heavy product expensive.
  • Cloudflare R2. Same API, no egress charge. For anything serving media this is usually the decision that matters.
  • Backblaze B2. Cheap storage, free egress through the Cloudflare partnership.
  • MinIO. Runs on your machine in a container. For development, so that nothing is uploaded to a real bucket from a laptop.
# One set of variables, four providers.
STORAGE_ENDPOINT=https://ACCOUNT.r2.cloudflarestorage.com # R2
# STORAGE_ENDPOINT=http://localhost:9002 # MinIO locally
STORAGE_REGION=auto
STORAGE_BUCKET=myapp
STORAGE_ACCESS_KEY=...
STORAGE_SECRET_KEY=...

Non-AWS endpoints generally need path-style addressing, which is one option on the client and the cause of most "works against S3, fails against MinIO" reports:

s3.NewFromConfig(cfg, func(o *s3.Options) {
o.BaseEndpoint = aws.String(endpoint)
o.UsePathStyle = true // bucket in the path, not in the hostname
})

When it goes wrong

SignatureDoesNotMatch

The request did not match what was signed. Almost always a header: a Content-Type the browser changed, or an extra header added by a fetch wrapper. Send exactly the headers that were signed and nothing else. Also check the clock: a server more than fifteen minutes out produces signatures everything rejects.

403 Forbidden on the PUT, with a valid URL

Usually the URL expired, which is five minutes by default and is easy to exceed while debugging. Otherwise the credentials lack s3:PutObject on that prefix: a presigned URL carries your permissions, so it can never grant more than the signer has.

CORS error on the PUT to the storage endpoint

The bucket has its own CORS configuration, separate from your API's. It must allow PUT from your origin.

[{
"AllowedOrigins": ["https://yourdomain.com"],
"AllowedMethods": ["PUT", "GET"],
"AllowedHeaders": ["content-type"],
"ExposeHeaders": ["etag"],
"MaxAgeSeconds": 3000
}]

413 Request Entity Too Large

A proxy rejected the body before your handler saw it. If you are uploading through your API, raise the limit on nginx with client_max_body_size, or whatever your platform calls it. If you are using presigned URLs and still see this, you are not using them for this particular upload.

The image shows locally and not on a phone

Stored URLs contain localhost, which on a phone means the phone. Store the key and build the URL at read time from configuration, so one record works from the browser, the emulator and a real device. If you want MinIO reachable from a phone on the same network, publish it on the machine's LAN address rather than 127.0.0.1.

As a field type

Grit treats this as a property of a field rather than a subsystem to assemble:

Terminal
$grit generate resource Product \
--fields "name:string:required,photo:file:image,manual:file,gallery:files:image"

That writes the presign and confirm endpoints, the type and size checks, the upload component with its progress and drag-and-drop, the background job that makes renditions, and the admin preview. STORAGE_DRIVER selects S3, R2, B2, MinIO or the local disk, and local disk means a new project uploads files before you have any cloud account at all.

The file storage reference covers the drivers and the field types.

Frequently asked questions

Presigned URLs or upload through my API?
Presigned for anything a user chooses from their device, which is nearly everything. Through your API when the file is small and you must inspect or transform it before it is stored at all, or when you are generating the file yourself server-side, where there is no browser involved.
How do I stop someone uploading something enormous?
Bind ContentLength into the signature, which makes storage itself reject a larger body. Then check the real size at confirm time and delete the object if it is wrong. The client-side check is a courtesy to honest users; the signature is the enforcement.
Is a random object key enough to keep a file private?
No. It makes the URL unguessable, which is worth something, and it is not permission: links are pasted into chats, forwarded and occasionally indexed. If the answer to "may this person see this" can be no, the bucket is private and your handler decides on each request.
Where should thumbnails be generated?
In a background job. Resizing a large photograph takes real CPU time, and doing it in the request means the user waits and your handler holds a worker. Record the upload immediately, queue the work, and let the interface show the original or a placeholder until the renditions appear.

Next steps