We are using AWS c3.8xlarge instance[32 core processor with 60 GB RAM] for one of the batch job execution.
We are running the same job[2 instances] to find out CPU utilization. The input file volume is huge (150 GB * 2 = 300 GB = 2 Billion total records)
When we run the job as single instance its completed in 2 hours.
But when we try to run the job parallel it's taking around 6 hours[The main reason is it's not utilizing the CPU at some point, it's using less than 10% at some point].
Is anyone else faced the kind of issues?
Any help on this greatly appreciated.
Regards, Shankar
Aucun commentaire:
Enregistrer un commentaire