Celery Workers Crash with Large Redis Queue

I have a celery app that processes a large number of messages from a self hosted Redis server. Recently the processes that feed the redis queue have needed to put a lot of messages in the queue (over 1 million messages). The issue I have is if I increase the concurrency of celery to anything over 1 they crash in less than a minute. I had this issue previously and was able to adjust the buffer limits on the Redis server (values below). Now with so many messages the issue has come back. Any ideas on how to fix this would be appreciated.

client-output-buffer-limit slave 512mb 256mb 60
client-output-buffer-limit pubsub 256mb 128mb 60

Also, everything is hosted in AWS on large EC2 instances.