[DynamoDB] software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: 에러 해결

Error

아래 코드를 통해 SpringBoot에서 DynamoDB를 통해 다량의 데이터를 BatchWrite할 경우 아래와 같은 에러가 발생

        DynamoDbAsyncTable<T> table = enhancedAsyncClient.table(tableName, TableSchema.fromBean(_class));
 
        return Flux.fromIterable(list)
                .buffer(25)
                .flatMap(spList-> {
                    log.info("length : {} ", spList.size());
                    BatchWriteItemEnhancedRequest bwier = BatchWriteItemEnhancedRequest.builder()
                            .writeBatches(spList.stream().map(item -> WriteBatch.builder(_class)
                                            .mappedTableResource(table)
                                            .addPutItem(item)
                                            .build())
                                    .toList())
                            .build();
                    return Mono.fromFuture(enhancedAsyncClient.batchWriteItem(bwier));
                })
                .then();

 

 

log

software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Acquire operation took longer than the configured maximum time. This indicates that a request cannot get a connection from the pool within the specified maximum time. This can be due to high request rate.
Consider taking any of the following actions to mitigate the issue: increase max connections, increase acquire timeout, or slowing the request rate.
Increasing the max connections can increase client throughput (unless the network interface is already fully utilized), but can eventually start to hit operation system limitations on the number of file descriptors used by the process. If you already are fully utilizing your network interface or cannot further increase your connection count, increasing the acquire timeout gives extra time for requests to acquire a connection before timing out. If the connections doesn't free up, the subsequent requests will still timeout.
If the above mechanisms are not able to fix the issue, try smoothing out your requests so that large traffic bursts cannot overload the client, being more efficient with the number of times you need to call AWS, or by increasing the number of hosts sending requests.
    at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) ~[sdk-core-2.24.1.jar:na]

 

반응형

 

---

 

Solution

원인 : connectionAcquisitionTimeout

 

request maximum time 보다 시간이 오래 걸려서 발생함.

 

아래 config 설정을 통해 maximum time 길게 설정하면 해결됨.

 

@Bean
public DynamoDbAsyncClient dynamoDbAsyncClient() {
    return DynamoDbAsyncClient.builder()
            .httpClientBuilder(NettyNioAsyncHttpClient.builder()
                    .connectionAcquisitionTimeout(Duration.ofMillis(5_000_000))
                    .connectionTimeout(Duration.ofMillis(5_000_000))
                    .maxConcurrency(100)
                    .tlsNegotiationTimeout(Duration.ofMillis(3_500_000)))
            .region(Region.of(region))
            .endpointOverride(URI.create(dynamoDBEndpoint))
            .credentialsProvider(StaticCredentialsProvider.create(
                    AwsBasicCredentials.create(accessKey, secretKey)))
            .build();
}

 

 

---

Thank you!