If this is a case of using Lambda entirely the wrong way, please let me know.
I want to install Scrapy into a Lambda function and invoke the function to begin a crawl. My first problem is how to install it, so that all of the paths are correct. I installed the program using the directory to be zipped as its root, so the zip contains all of the source files and the executable. I am basing my efforts on this article. In the line it says to include at the beginning of my function, where does the "process" variable come from? I have tried,
var process = require('child_process');
var exec = process.exec;
process.env['PATH'] = process.env['PATH'] + ':' +
process.env['LAMBDA_TASK_ROOT']
but I get the error,
"errorMessage": "Cannot read property 'PATH' of undefined",
"errorType": "TypeError",
Do I need to include all of the library files, or just the executable from /usr/lib ? How do I include that one line of code the article says I need?
Edit: I tried moving the code into a child_process.exec, and received the error
"errorMessage": "Command failed: /bin/sh: process.env[PATH]: command not found\n/bin/sh: scrapy: command not found\n"
Here is my current, entire function
console.log("STARTING");
var process = require('child_process');
var exec = process.exec;
exports.handler = function(event, context) {
//Run a fixed Python command.
exec("process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT']; scrapy crawl backpage2", function(error, stdout) {
console.log('Scrapy returned: ' + stdout + '.');
context.done(error, stdout);
});
};
Aucun commentaire:
Enregistrer un commentaire