In a
previous post I had described the new AWS SDK for Java 2 which provides non-blocking IO support for Java clients calling different AWS services. In this post I will go over an approach that I have followed to unit test the AWS DynamoDB calls.
There are a few ways to spin up a local version of DynamoDB -
1. AWS provides a
DynamoDB local
2.
Localstack provides a way to spin up a good number of AWS services locally
3. A docker version of
DynamoDB Local
4.
Dynalite, a node based implementation of DynamoDB
Now to be able to unit test an application, I need to be able to start up an embedded version of DynamoDB using one of these options right before a test runs and then shut it down after a test completes. There are three approaches that I have taken:
1. Using a
JUnit 5 extension that internally brings up a AWS DynamoDB Local and spins it down after a test.
2. Using
testcontainers to start up a docker version DynamoDB Local
3. Using
testcontainers to start up
DynaLite
JUnit5 extension
JUnit5 extension provides a convenient hook point to start up an
embedded version of DynamoDB for tests. It works by pulling in a version of DynamoDB Local as a maven dependency:
1 2 3 4 5 | dependencies {
...
testImplementation("com.amazonaws:DynamoDBLocal:1.11.119")
...
}
|
A complication with this dependency is that there are native components (dll, .so etc) that the DynamoDB Local interacts with and to get these in the right place, I depend on a Gradle task:
1 2 3 4 5 6 7 8 9 10 11 12 13 | task copyNativeDeps(type: Copy) {
mkdir "build/native-libs"
from(configurations.testCompileClasspath) {
include '*.dll'
include '*.dylib'
include '*.so'
}
into 'build/native-libs'
}
test {
dependsOn copyNativeDeps
}
|
which puts the native libs in build/native-libs folder, and the extension internally sets this path as a system property:
1 | System.setProperty("sqlite4java.library.path", libPath.toAbsolutePath().toString())
|
Here is the codebase to the JUnit5 extension with all these already hooked up - https://github.com/bijukunjummen/boot-with-dynamodb/blob/master/src/test/kotlin/sample/dyn/rules/LocalDynamoExtension.kt
A test using this extension looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | class HotelRepoTest {
companion object {
@RegisterExtension
@JvmField
val localDynamoExtension = LocalDynamoExtension()
@BeforeAll
@JvmStatic
fun beforeAll() {
val dbMigrator = DbMigrator(localDynamoExtension.syncClient!!)
dbMigrator.migrate()
}
}
@Test
fun saveHotel() {
val hotelRepo = DynamoHotelRepo(localDynamoExtension.asyncClient!!)
val hotel = Hotel(id = "1", name = "test hotel", address = "test address", state = "OR", zip = "zip")
val resp = hotelRepo.saveHotel(hotel)
StepVerifier.create(resp)
.expectNext(hotel)
.expectComplete()
.verify()
}
}
|
The code can interact with a fully featured DynamoDB.
The JUnit5 extensions approach works well but it requires an additional dependency with native binaries to be pulled in. A cleaner approach may be to use the excellent
Testcontainers to spin up a docker version of DynamoDB Local the following way:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | class HotelRepoLocalDynamoTestContainerTest {
@Test
fun saveHotel() {
val hotelRepo = DynamoHotelRepo(getAsyncClient(dynamoDB))
val hotel = Hotel(id = "1", name = "test hotel", address = "test address", state = "OR", zip = "zip")
val resp = hotelRepo.saveHotel(hotel)
StepVerifier.create(resp)
.expectNext(hotel)
.expectComplete()
.verify()
}
companion object {
val dynamoDB: KGenericContainer = KGenericContainer("amazon/dynamodb-local:1.11.119")
.withExposedPorts(8000)
@BeforeAll
@JvmStatic
fun beforeAll() {
dynamoDB.start()
}
@AfterAll
@JvmStatic
fun afterAll() {
dynamoDB.stop()
}
fun getAsyncClient(dynamoDB: KGenericContainer): DynamoDbAsyncClient {
val endpointUri = "http://" + dynamoDB.getContainerIpAddress() + ":" + dynamoDB.getMappedPort(8000)
val builder: DynamoDbAsyncClientBuilder = DynamoDbAsyncClient.builder()
.endpointOverride(URI.create(endpointUri))
.region(Region.US_EAST_1)
.credentialsProvider(StaticCredentialsProvider
.create(AwsBasicCredentials
.create("acc", "sec")))
return builder.build()
}
...
}
}
|
This code starts up DynamoDB at a random unoccupied port and provides this information so that the client can be created using this information. There is a little Kotlin workaround that I had to do based on an issue reported here - https://github.com/testcontainers/testcontainers-java/issues/318
Dynalite is a javascript based implementation of DynamoDB and can be run for tests again using the TestContainer approach. This time however there is already a
TestContainer module for Dynalite. I found that it does not support JUnit5 and sent a
Pull request to provide this support, in the iterim the raw docker image can be used and this is how a test looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | class HotelRepoDynaliteTestContainerTest {
@Test
fun saveHotel() {
val hotelRepo = DynamoHotelRepo(getAsyncClient(dynamoDB))
val hotel = Hotel(id = "1", name = "test hotel", address = "test address", state = "OR", zip = "zip")
val resp = hotelRepo.saveHotel(hotel)
StepVerifier.create(resp)
.expectNext(hotel)
.expectComplete()
.verify()
}
companion object {
val dynamoDB: KGenericContainer = KGenericContainer("quay.io/testcontainers/dynalite:v1.2.1-1")
.withExposedPorts(4567)
@BeforeAll
@JvmStatic
fun beforeAll() {
dynamoDB.start()
val dbMigrator = DbMigrator(getSyncClient(dynamoDB))
dbMigrator.migrate()
}
@AfterAll
@JvmStatic
fun afterAll() {
dynamoDB.stop()
}
fun getAsyncClient(dynamoDB: KGenericContainer): DynamoDbAsyncClient {
val endpointUri = "http://" + dynamoDB.getContainerIpAddress() + ":" + dynamoDB.getMappedPort(4567)
val builder: DynamoDbAsyncClientBuilder = DynamoDbAsyncClient.builder()
.endpointOverride(URI.create(endpointUri))
.region(Region.US_EAST_1)
.credentialsProvider(StaticCredentialsProvider
.create(AwsBasicCredentials
.create("acc", "sec")))
return builder.build()
}
...
}
}
|
Conclusion
All of the approaches are useful in being able to test integration with DynamoDB. My personal preference is using the TestContainers approach if a docker agent is available else with the JUnit5 extension approach. The samples with fully working tests using all the three approaches are available in my github repo - https://github.com/bijukunjummen/boot-with-dynamodb
