Please forgive me if this appears twice, the forum claimed to discard my last message after successful preview and an attempt to post it.
I have an oozie custom action on the Cloudera Quickstart 5.4.x VM, with an associated main class. The main class is in scala (for intereoperability with a framework), and I'm having some challenges during testing. Before going too deep into the details I would like to ask the following questions:
What is the recommended approach for the custom action executor to resolve the class name of the main class, particularly if the main class is written in scala?
What is the automated and recommended way to determine what (if any) dependencies must be provided with (i) the custom action and (ii) the main class?
What is the recommended way to package the dependencies (e.g. put extra jars in the directory, use a shadow/fat jar)?
The details follow for a simplified sanitized version of what I'm dealing with that shows the salient features.
First, let's consider the custom action executor, whch has the following structure, please note the labeled problematic line impacted at runtime by the presence/absence of dependencies.
The MyScalaShellMain class is a scala class with a companion object (to support the static main method), with the following structure:
If I build this in the same jar and install it in both the /var/lib/oozie directory and the /lib directory next to my workflow.xml as a shared lib, I get a ClassNotFoundException, rethrown as a NoClassDefFoundError, pointing to indicated line in the custom action executor, but complaining about com/typesafe/scalalogging/Logging (is a static method invoking the constructor here?).
So I tried to resolve the depenencies by including the following jars in the /var/lib/oozie directory and the lib directory next to my workflow.xml as a shared lib.
scala-library-2.10.5.jar
scala-logging-slf4j_2.10-2.1.2.jar
scala-logging-api_2.10-2.1.2.jar
(The versions are selected to be compatible with the framework used for the production main).
Fortunately the workflow runs with a successful completion status, jowever now the logging of the oozie workflow is no longer populated.
I think that is due to dependency conflicts, and am not sure how to correctly resolve them.