lundi 27 avril 2015

Redis design help (From Relational to NoSQL)

I am very much a SQL developer and am new to redis, but it's performance is very interesting. I have a problem I think redis could help me very much in. I have a SQL table familiar to this:

| CONTAINER <String><NoUnq> | PROCESS <String><NoUnq> | PROCESS_DATA <String><NoUnq> | TimeCreated <TimeStamp><NoUnq>|

This table when populated to its max has roughly ~450,000,000 rows. I am running this on AWS. With these rows I select all the processes within a container (~1,000,000 containers), so I would have something like this in sql (of course container is indexed):

SELECT * FROM table WHERE container = '[CONTAINER_NAME]';

I then have a cronjob script which runs every hour and removes old processes from containers with something like this:

DELETE FROM table WHERE TimeCreated <= [SOME_TIME];

So essentially I like to have processes which are not older than ~4-5 hours. Looking at Redis I feel like I can engineer something similar to improve my performance, but am having trouble to convert this SQL like design into Redis.

My first thought was to use HSET, but I found out HSET does not allow the EXPIRE command on fields so I could not automatically remove old process. I am most concerned about performance and efficiency.




Aucun commentaire:

Enregistrer un commentaire