jeudi 4 juin 2015

How to automate Amazon aws EC2 for scraping

Hi I'd like to set up some amazon EC2 instances (multiple) to scrape data from arbitrary sites. The way I imagine it being set up is one amazon instance that's a master which will programatically set up other instances to scrape. Right now I have php scripts that can scrape the way I want it to, but how can I set up my master server to...

1) make other ec2 instances

2) communicate between the master server and slave servers




Aucun commentaire:

Enregistrer un commentaire