jeudi 1 octobre 2015

Flume Kafka throughput slow down

We have few flume machines from US-west (aws) & US-east talking to three node kafka cluster (all in US -East) .

We see that throughput of US East machines are like atleast 400 (in sync mode -Flume settings).
we see that throughput of US West machines are 12 (in sync mode -Flume settings).

We can see that the problem is not with Bandwidth .. (Reasoning for this statement:)
1) Type of AWS instance .kafka -(m3.xlarge) & flume (m1.xlarge)
2) We can see that US east machine can go till 400 when its switched to async .

Can some one help in troubleshooting this ?




Aucun commentaire:

Enregistrer un commentaire