mercredi 5 août 2015

Amazon Elasticache Failover

We have been using AWS Elasticache for about 6 months now without any issues. Every night we have a Java app that runs which will flush DB 0 of our redis cache and then repopulate it with updated data. However we had 3 instances between July 31 and August 5 where our DB was successfully flushed and then we were not able to write the new data to the database.

We were getting the following exception in our application:

redis.clients.jedis.exceptions.JedisDataException: redis.clients.jedis.exceptions.JedisDataException: READONLY You can't write against a read only slave.

When we look at the cache events in Elasticache we can see

Failover from master node prod-redis-001 to replica node prod-redis-002 completed

We have not been able to diagnose the issue and since the app was running fine for the past 6 months I am wondering if it is something related to a recent Elasticache release that was done on the 30th of June. http://ift.tt/1DvpkEY

We have always been writing to our master node and we only have 1 replica node.

If someone could offer any insight it would be much appricated.




Aucun commentaire:

Enregistrer un commentaire