Benchmark methodology

How the Grit side of the benchmark is built: every step, nothing hidden

The Grit application is generated, not hand-written. That is the whole claim being tested: the code the CLI emits, unmodified except for making the routes public, is what gets measured.

How this framework is kept from being handicapped

A benchmark is only worth publishing if the loser was given every reasonable advantage. These are the specific decisions made for this framework, and each one is worth saying out loud on camera.

Generated with `grit new` and `grit generate resource`, then left alone. The only edit is moving the product routes out of the authenticated group.
GIN_MODE=release, and Studio, Pulse and Sentinel all switched off: each would be work the other frameworks are not doing.
REDIS_URL empty, which as of v3.132.0 genuinely disables cache, jobs, worker and cron.
No auth on the benchmarked routes. With a token in play, part of what you measure is JWT parsing rather than the request path, and that is true for every framework here, so none of them have it.

Reproduce it, start to finish

Install the two tools you need

Docker runs every app and the database. k6 generates the load. Nothing else is required: you do not need Go, PHP, Python, Node or Bun installed locally, because every framework builds inside a container.

# macOS
brew install k6
# Windows
winget install k6 --source winget
# Linux
sudo gpg -k && sudo gpg --no-default-keyring \
  --keyring /usr/share/keyrings/k6-archive-keyring.gpg \
  --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" \
  | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update && sudo apt-get install k6

k6 version
docker --version

2
Clone the harness
Everything below lives in the benchmarks directory of the Grit repository: the compose file, the k6 script, the seed data, and each framework’s application.
```
git clone https://github.com/MUKE-coder/grit.git
cd grit/benchmarks
```
3
Start Postgres and create the databases
One Postgres instance is shared by every framework, tuned once, with 8 CPUs: deliberately more than any application gets, so the database is never what gives out first. Each framework gets its own database so their schemas cannot collide.
```
docker compose up -d postgres

for db in bench_grit bench_encore bench_bun bench_express bench_nextjs bench_laravel bench_django; do
  docker compose exec -T postgres psql -q -U bench -d bench -c "CREATE DATABASE $db;"
done
```
4
Create the table and load identical rows
This is the single most important step for a fair result. Every framework gets the same table, created from the same SQL file, holding the same 10,000 rows with the same UUIDs. Letting each ORM run its own migration would mean comparing schemas, not frameworks: Eloquent, Django and GORM disagree about integer widths, timestamp precision and which columns get indexed.
```
for db in bench_grit bench_encore bench_bun bench_express bench_nextjs bench_laravel bench_django; do
  docker compose exec -T postgres psql -q -U bench -d $db < seed/schema.sql
  docker compose exec -T postgres psql -q -U bench -d $db < seed/products.sql
done

# every one should print 10000
for db in bench_grit bench_encore bench_bun bench_express bench_nextjs bench_laravel bench_django; do
  docker compose exec -T postgres psql -tA -U bench -d $db -c "SELECT count(*) FROM products;"
done
```
If any of these prints something other than 10000, stop and fix it. A benchmark where one side has more rows than another is measuring the row count.
5
Generate the application
Two commands. The resource generator writes the model, service, handler, routes and tests; nothing below modifies the handler it produces.
```
grit new grit-bench --api
cd grit-bench

grit generate resource Product \
  --fields "name:string,sku:string,description:text,price:float,stock:int,active:bool"
```
6
Make the routes public
The generator puts new resources behind auth, which is the right default and the wrong thing for a benchmark: with a token in play you are partly measuring JWT parsing. Move the five product routes out of the `protected` group into their own public group in internal/routes/routes.go.
```
// Benchmark: Product CRUD with no auth, so a load test measures the
// framework's request path rather than JWT parsing.
products := v1.Group("/products")
{
    products.GET("", productHandler.List)
    products.GET("/:id", productHandler.GetByID)
    products.POST("", productHandler.Create)
    products.PUT("/:id", productHandler.Update)
    products.DELETE("/:id", productHandler.Delete)
}
```
Also delete the generated `admin.DELETE("/products/:id", ...)` line. Registering the same method and path twice makes Gin panic at startup.

Build and start it

The scaffold ships its own multi-stage Dockerfile; nothing is added to it.

cd ..
docker compose build grit
docker compose up -d grit

# should print exactly one line about Redis being disabled, and no dial errors
docker compose logs grit | head -5

8
Check every framework is on an ORM, not raw SQL
This is the fairness decision that matters most after the shared schema. Grit’s generated handlers use GORM and you cannot swap that out, so measuring any framework against hand-written SQL compares an ORM to no ORM, which flatters that framework for a reason that has nothing to do with it. Express on raw pg measured 2,000 req/s on writes; the same Express on Prisma measured 773. Same framework, same machine, same test.
```
# Grit      GORM          apps/api/internal/models/product.go
# Laravel   Eloquent      laravel-bench/app/Models/Product.php
# Django    Django ORM    django-bench/products/models.py
# Express   Prisma        express-bench/prisma/schema.prisma
# Next.js   Prisma        nextjs-bench/prisma/schema.prisma
# Bun       Drizzle       bun-bench/schema.ts
# Encore    Drizzle       encore-bench/products/schema.ts

grep -rl "prisma|drizzle|Eloquent|models.Model"   express-bench nextjs-bench bun-bench encore-bench django-bench laravel-bench 2>/dev/null | head
```
If you swap any of these for raw SQL the numbers move enough to change the conclusion. That is the single easiest way to make this benchmark say whatever you want it to say.
9
Prove the two apps return the same bytes
Before measuring anything, confirm the framework you are testing returns exactly what Grit returns for the same record. If the payloads differ, say a missing field or a price as a string instead of a number, then the two are doing different amounts of work and the comparison is void.
```
ID=$(python -c "import json;print(json.load(open('seed/ids.json'))[0])")

for app in grit grit; do
  echo "--- $app"
  docker run --rm --network benchmarks_default curlimages/curl -s \
    "http://$app:8080/api/v1/products/$ID"
  echo
done
```
This is the step most benchmarks skip, and it is the one that decides whether the rest means anything. Show it on camera.
10
Run the benchmark
Three repetitions of four scenarios, one application at a time with the others stopped. Each run resets the dataset first, discards a warm-up, and samples container CPU while the load is actually running.
```
APPS="grit grit" REPS=3 ./final.sh

python aggregate.py
```

Reading your own results

Grit saturates its own container on single-row reads and writes, so those are true ceilings. On the paginated list it sits far below its CPU limit while Postgres is pinned: that scenario is bounded by the database, so the number is a floor and the real gap is wider than the chart shows.

aggregate.py prints the app container's CPU and Postgres's CPU next to every row. That pair is what tells you whether a number is a real ceiling. If the app is pinned near 400% then you are seeing its limit. If the app is idling while Postgres is near 800%, the database gave out first. That row is a floor, the framework would go faster on a bigger database, and quoting it as “X does N req/s” overstates what you measured.