I'm using Kinesis to store records sent from Jmeter installed on EC2 server. The problem is that when I launch 7200 threads and use 1 shared on my Kinesis stream, all works fine. If I launch 9000 threads I receive this error
Rate exceeded for shard shardId-000000000001 in stream Jmeter under account 769870455028. (Service: AmazonKinesis; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: 98f687d9-ffbe-11e4-a897-357ee8c24764)
So I increse the number of shard, setting it to 2 and 3 but it doesn't work. So I think that the problem isn't the shard number but my java code or i don't know what else. This is my code:
public MyKinesisClient( String streamName, int partitionKey, String accessKey, String secretKey, String endpoint, String serviceName, String regionId ) {
this.streamName=streamName;
this.partitionKey=partitionKey;
AWSCredentials credentials = null;
credentials = new BasicAWSCredentials(accessKey, secretKey);
kinesisClient = new AmazonKinesisClient(credentials);
kinesisClient.setEndpoint(endpoint,serviceName,regionId);
}
/**
* Metodo utilizzato per l'invio di un json a Kinesis
* @param json: com.amazonaws.util.json.JSONObject da inviare a Kinesis
* @throws UnsupportedEncodingException
* @throws JSONException
*/
public void sendJson(JSONObject json) throws UnsupportedEncodingException, JSONException {
try{
PutRecordRequest putRecordRequest = new PutRecordRequest();
putRecordRequest.setStreamName(streamName);
putRecordRequest.setData(ByteBuffer.wrap(json.toString().getBytes("utf-8")));
//putRecordRequest.setData(ByteBuffer.wrap(String.format("testData-%d", createTime).getBytes()));
putRecordRequest.setPartitionKey(String.format("partitionKey-%d", partitionKey));
kinesisClient.putRecord(putRecordRequest);
}catch(Exception e){
System.out.println(e.getMessage());
}
}
There is an instruction to use more shard?Thanks in advance
Aucun commentaire:
Enregistrer un commentaire