Creating Custom Spark

After you have created your jar file running sbt assembly, you can create your Custom Spark cofiguration

Click on the + icon at the top right


Select Spark

Select Custom Spark

Enter Title , Description. Select the Spark Cluster from the dropdown. Select Main Class from the dropdown.

In the Aruguments tab, add the input and output datasheets.Click the + icon to add a datasheet. On hover select datasheet icon appears. Select the datasheet by clicking this icon.

In the Memory tab, edit the default parameters if required


In the Model Arguments tab, enter all the parameters


Save the configuration by clicking the SAVE button

Note: Special characters maybe included in the spark configuration of the Docube. Also, an API call needs to be passed with the path instead of ID.