Skip to content

About

A sample app for the MitsuruTech Learning Platform: upload a file, S3 keeps the bytes, Postgres remembers where they are. In Java (Spring Boot).

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

File uploads in Java: S3 for the bytes, Postgres for everything else

English | Français | 日本語 | Español

A Java (Spring Boot) sample app for the MitsuruTech Learning Platform. Upload a file; S3 keeps it; Postgres remembers where it went and what it is called. The table on the page comes out of Postgres, and every download comes out of S3.

It is deliberately small, four routes and four source files, so you can read the whole thing before you change any of it.

src/main/java/dev/learntools/uploads/
  Application.java   Spring Boot's entry point
  Web.java           the web: four routes and nothing else
  Store.java         the two stores, and the order they are written in
  Page.java          the HTML

Deploying it here

Open Deploy on the platform, pick this repository, and start a database and the cloud stack alongside it (how deploying from GitHub works). Everything the app needs is then in its environment already; there is nothing to configure. The platform builds it with mvn -q -DskipTests package and starts it with java -jar target/*.jar.

Variable Set by What it is
DATABASE_URL your session the Postgres server running as db
AWS_ENDPOINT_URL your session the cloud stack running as cloud
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION your session credentials for it
PORT the platform the port to listen on
BUCKET_NAME you, optionally defaults to uploads

The three things worth reading

1. forcePathStyle(true), in Store.java. This is the setting everybody misses first. By default the AWS SDK puts the bucket in the hostname, uploads.cloud, and there is no such name on your session's network, so every call fails with a DNS error that says nothing about buckets. Path style asks for http://cloud:4566/uploads/... instead.

2. The app makes its own bucket. Your cloud stack starts completely empty, exactly as a real AWS account you just opened would. createBucket is your code's job, it runs on every start, and it catches "you already own this" so a restart is not a crash.

3. Write S3 first, then the row. Delete the row first, then S3.

  • Upload the bytes, then insert the row. If the insert fails you have an object nobody points at: invisible, harmless, sweepable.
  • Write the row first and a failed upload leaves a file on the page that cannot be opened, and no way to tell that row from a good one.
  • Deleting runs the other way for the same reason.

One of those failures is a few wasted bytes. The other is a broken page. Always let the thing that can be orphaned harmlessly go first.

A Java detail worth knowing: DATABASE_URL looks like postgres://user:password@host:5432/name, and the Java database driver cannot read that. Store.java takes it apart into a jdbc:postgresql:// address plus a user and a password, which is the shape JDBC wants.

Two safety habits, taken seriously in a sample on purpose

The S3 key is a uuid, never the filename. Two people upload notes.txt and one must not overwrite the other; and a name that came from a browser is not a thing to build a path from: ../../etc/passwd is a perfectly valid filename as far as a form is concerned. The real name is a column, which is where it is safe.

Downloads are always attachment, always application/octet-stream, always nosniff. Anybody can upload anything. An HTML file served inline would run as a page on your app's own address, with whatever that page can reach; saved to disk it is inert.

And every filename is escaped on the way back onto the page: a file called <img src=x onerror=alert(1)> is a perfectly good filename and must not become markup.

Running it on your own machine

You need Java 21, Maven, a Postgres and something S3-shaped. With Docker:

docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=dev --name pg postgres:16
docker run -d -p 4566:4566 --name cloud localstack/localstack

export DATABASE_URL=postgres://postgres:dev@localhost:5432/postgres
export AWS_ENDPOINT_URL=http://localhost:4566
export AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_REGION=us-east-1

mvn -q -DskipTests package && java -jar target/app.jar

Then open http://localhost:8080. To run the tests (they check that the page is built exactly as expected), use mvn test.

Things to try next

  • Stream a large upload straight to S3 instead of holding it in memory (Web.java says where).
  • Add a download count, in DynamoDB: your cloud stack emulates that too.
  • Put the files behind a login, so the table is per person.
  • Show an image preview, using a pre-signed URL rather than proxying the bytes.

Licence

This work is licensed under CC BY-NC-SA 4.0. You may share it and adapt it for learning and teaching, if you credit Mitsuru Technologies Ltd, do not use it commercially, and share anything you make from it under the same licence. The full text is in the LICENSE file.

About

Made by Mitsuru Technologies, who build the MitsuruTech Learning Platform for teaching programming. The same sample exists in other languages: github.com/mitsurutech/uploads-node, uploads-python and uploads-go.

Teaching a class? See the price or create a college account.

About

A sample app for the MitsuruTech Learning Platform: upload a file, S3 keeps the bytes, Postgres remembers where they are. In Java (Spring Boot).

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages