Spring Boot Native vs Go: A performance comparison
Introduction
In my previous post, I compared the performance of a simple REST controlled done with Quarkus and Spring boot. A few months, after I published that post, Spring launched a new project called Spring Native. The project it is still experimental but allows you to build native images with GraalVM. According to the Spring announcement, “Spring Native offers interesting characteristics including almost instant startup (typically < 100ms), instant peak performance and lower memory consumption at the cost of longer build times and fewer runtime optimizations than the JVM”.
During the time Spring Native was released, I was learning Go. The main reason why I was interested in Go was for its performance: Go is fast, has a low memory consumption and a low CPU usage.
The question I wanted to answer was: is Go Performance significantly better than Spring Boot (particularly now that a native option exists)?
Spring Boot Native Application
The Spring Boot application has been generated from https://start.spring.io/. It is a simple application using Java 11 that contains the following dependencies:
- Spring Boot version 2.4.5
- Spring Native
- Spring Web
- Spring Data JPA
- Spring Actuator
- Lombok
Image size: 172 MB
Image Build time: ~ 13 minutes
Please find the code here.
Go Application
The Go application is a very simple application build from scratch. It has been developed with Go version 1.16 and contains the following modules:
- Gin Web framework
- Gorm: An ORM library for Go
- Go Mysql driver
Image size: 14.5MB
Image build time: ~ 48 seconds
Please find the code here.
Infrastructure
This project uses Terraform for creating and deploying resources to AWS.
Both applications will be using exactly the same infrastructure:
- 1 Application Load Balancer (ALB)
- 1 AWS ECS cluster
- 2 tasks running in the cluster. Each task has memory = 512MiB and CPU = 256.
- A MySQL 8.0.17 engine managed by AWS RDS
For more information, please check the terraform folder which contains all the modules and resources used.
Rest Controller
The idea of this project was to build a simple API that implements create and read operations. Both applications have implemented the same endpoints:
- GET /products: Returns the last 20 products.
- GET /products/{id}: Returns one single product for a given Id.
- POST /products: Saves a new product.
Load tests
Load testing has been done with Gatling. Gatling is a great developer tool to load test web applications.
The load test tries to simulate a common use case. Each user will perform the following actions:
- Get all the latest products.
- Save a new product.
- Retrieve the product saved on step 2.
The test ramps up 200 users during the first minute and then keeps constantly 200 users for 2 hours.
Response time results
Resources results
Conclusion
Looking at the response time results from Gatling, the mean time in all 3 operations is faster in Go. The response time for saving one product and retrieving the last product is around 20% faster in Go. And when retrieving the 20 latest products it is 133% faster in Go than in Java and Spring Boot Native. I wonder how much the impact of ORM libraries (Gorm vs Hibernate) has on these results but clearly Go is significantly faster than Java and Spring Boot Native.
In terms of resources utilisation, Go is also the winner. Spring Native has clearly improved against the standard Spring Boot application which I compared in a previous post. The CPU and memory usage in the Go application is very impressive, especially taking into account the tests were hitting the API with 200 requests per second.
It is worth noticing that during the 2 hours that the load tests last, the CPU and memory were stable in the Go application whilst the memory usage in Spring Native keep increasing.
In my opinion, Spring Native it is a good first step to reducing memory consumption. Currently I still find the build image time is too long, although I expect that they will improve it in future releases. The Spring team will need to improve it, particularly now that many people use CI/CD tools like GitHub actions where you have to pay by minute.
On the other hand, I was surprised by how good the performance was with Go. I have spent most of my career writing Java and Spring applications so I am still new to the Go world. I don’t think Go will replace Java and there are important features missing in Go, like Generics (although the Go team are planning to add them soon). Nevertheless, I think Go could be a better fit for developing Microservices or for Function-as-a-Service (Serverless).
Wouldn’t the Java version be faster if you used WebFlux?
I’m not so sure. I’ve done tests similar to this mvc vs webflux with r2dbc. mvc could serve two times more requests per second than webflux. Maybe these kind of tests are not representative, because od r2dbc limitations, but still…
Just to add, I was pulling 10_000 rows at a time, not 20
it makes no sense to compare frameworks with a database in your pipeline. Most likely your test shows difference of databse connectivity & orm perfromance with the default configuration. For the fair comparison with a database in your pipeline you have to tune orm & connection pool to a similar settings with all nuances, which is non-trivial. The more close results would be to compare pure computations of minimal spring-native image vs go, but even in this case spring minimal provides way more functionality and java features (e.g. java generics are actively used, while in Go generics may cause performance degradation) etc.
Wow this is amazing! I just read an article that compared backend web server performance and said spring was the fastest based on their benchmarks…. But they left out GO! I though hmm…. Let’s see about this an sure enough go takes the cake!