vendredi 28 août 2015

EMR: Is the Spark application available at bootstrap time?

I am trying to start an EMR cluster with Spark using the CLI, where I specify Spark as an application. I also have some bootstrap scripts that configure things like IPython notebooks on top of Spark. However, when I try to refer to common Spark locations in my bootstrap scripts (/usr/bin/spark or /usr/lib/spark/bin) I get not found errors.

Can someone help me understand what the sequence of events in EMR clusters is -- are applications installed after bootstrapping?




Aucun commentaire:

Enregistrer un commentaire