vendredi 29 mai 2015

Put multiple items into DynamoDB by Java code

i would like use batchWriteItem method of SDK Amazon to put a lot of items into table. I retrive the items from Kinesis, ad it has a lot of shard. I used this method for one item:

public static void addSingleRecord(Item thingRecord) {


    // Add an item
    try
    {

        DynamoDB dynamo = new DynamoDB(dynamoDB); 
        Table table = dynamo.getTable(dataTable);
        table.putItem(thingRecord);

    } catch (AmazonServiceException ase) {
        System.out.println("addThingsData request  "
                + "to AWS was rejected with an error response for some reason.");
        System.out.println("Error Message:    " + ase.getMessage());
        System.out.println("HTTP Status Code: " + ase.getStatusCode());
        System.out.println("AWS Error Code:   " + ase.getErrorCode());
        System.out.println("Error Type:       " + ase.getErrorType());
        System.out.println("Request ID:       " + ase.getRequestId());
    } catch (AmazonClientException ace) {
        System.out.println("addThingsData - Caught an AmazonClientException, which means the client encountered "
                + "a serious internal problem while trying to communicate with AWS, "
                + "such as not being able to access the network.");
        System.out.println("Error Message: " + ace.getMessage());
    }
}

public static void addThings(String thingDatum) {
    Item itemJ2;
    itemJ2 = Item.fromJSON(thingDatum);
    addSingleRecord(itemJ2);

}

The item is passed from:

private void processSingleRecord(Record record) {
    // TODO Add your own record processing logic here


    String data = null;
    try {


        // For this app, we interpret the payload as UTF-8 chars.
        data = decoder.decode(record.getData()).toString();
        System.out.println("**processSingleRecord - data " + data);
        AmazonDynamoDBSample.addThings(data);

    } catch (NumberFormatException e) {
        LOG.info("Record does not match sample record format. Ignoring record with data; " + data);
    } catch (CharacterCodingException e) {
        LOG.error("Malformed data: " + data, e);
    }
}

Now if i want to put a lot of record, I will use:

// Add a new item to Forum
        TableWriteItems dataTableWriteItems = new TableWriteItems(dataTable) //Forum
            .withItemsToPut(thingRecord);
        System.out.println("Making the request.");
        BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem(batchWriteItemRequest);

but in the amazon sample it used two table with one item, so this method is wrong. How can I group the item before send them? I have to look carefully because I have a lot of shard and then a lot of thread.




Aucun commentaire:

Enregistrer un commentaire