I use python's boto.kinesis module to write records to AWS Kinesis. The maximum throughput that is reached is about 40 puts/sec. However, according to the Kinesis FAQ:
Each shard can support up to 1000 PUT records per second.
So my current approach reaches only 4% what is theoretically possible, which seems terribly low.
Does anyone have an idea how the throughput can be improved?
Setup: The Kinesis Stream is an instance with one shard. The producer is on a dedicated AWS EC2 instance (t3.medium) in the same region as the Kinesis Stream. It creates strings of about 20 characters lengths and sends them to the Kinesis Stream via boto.kinesis.Connection.put_record("my_stream", my_message).
Simplified code:
from boto import kinesis
import time
connection = kinesis.connect_to_region(REGION)
stream = connection.create_stream("my_stream", shard_count=1)
time.sleep(60) # wait a minute until stream is created
for i in range NUM_MESSAGES:
my_message = "This is message %d" % i
connection.put_record(my_message, "my_stream", "partition_key")
Aucun commentaire:
Enregistrer un commentaire