[DynamoDB] reactor.core.Exceptions$ErrorCallbackNotImplemented: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Too many items requested for the BatchWriteItem call 에러 해결

Error

아래 코드와 같이 SpringBoot에서 DynamoDbAsyncTable를 이용하여 DynamoDb에 batchWrite할 경우 에러가 발생함

        DynamoDbAsyncTable<T> table = enhancedAsyncClient.table(tableName, TableSchema.fromBean(_class));
 
        BatchWriteItemEnhancedRequest bwier = BatchWriteItemEnhancedRequest.builder()
                .writeBatches(list.stream().map(item -> WriteBatch.builder(_class)
                                .mappedTableResource(table)
                                .addPutItem(item)
                                .build())
                        .toList())
                .build();
 
        return Mono.fromFuture(enhancedAsyncClient.batchWriteItem(bwier)).then();

reactor.core.Exceptions$ErrorCallbackNotImplemented: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Too many items requested for the BatchWriteItem call (Service: DynamoDb, Status Code: 400, Request ID: afa677fd-7d67-4184-b375-4ca0b330321c)
Caused by: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Too many items requested for the BatchWriteItem call (Service: DynamoDb, Status Code: 400, Request ID: afa677fd-7d67-4184-b375-4ca0b330321c)
    at software.amazon.awssdk.services.dynamodb.model.DynamoDbException$BuilderImpl.build(DynamoDbException.java:104) ~[dynamodb-2.24.1.jar:na]

 

반응형

 

---

 

Solution

BatchWriteItem 사용 시, 네트워크를 통해 최대 16MB의 데이터를 전송가능, 최대 25개 항목 넣기 또는 삭제 작업 구성, 개별 항목은 일단 저장되면 최대 400KB까지 가함

 

참고 : https://docs.aws.amazon.com/ko_kr/amazondynamodb/latest/APIReference/API_BatchWriteItem.html

 

아래처럼 25개씩 Flux를 분할하면 실행

 

        DynamoDbAsyncTable<T> table = enhancedAsyncClient.table(tableName, TableSchema.fromBean(_class));
 
        return Flux.fromIterable(list)
                .buffer(25)
                .flatMap(spList-> {
                    log.info("length : {} ", spList.size());
                    BatchWriteItemEnhancedRequest bwier = BatchWriteItemEnhancedRequest.builder()
                            .writeBatches(spList.stream().map(item -> WriteBatch.builder(_class)
                                            .mappedTableResource(table)
                                            .addPutItem(item)
                                            .build())
                                    .toList())
                            .build();
                    return Mono.fromFuture(enhancedAsyncClient.batchWriteItem(bwier));
                })
                .then();

 

 

---

Thank you!