Custom Python Models¶
Dataiku DSS provides useful and powerful built-in visual tools, but also always allows you to extend its functionality with code.
This article explains how to add custom Python code-based machine learning (ML) algorithms in our guided Machine Learning framework. You are advised to read the tutorial Your first predictive model prior to starting this reading.
Specifying the Custom Model¶
In a Visual Analysis (Lab), go to the Models tab. In the Design stage of a model, go to the Algorithms panel. The list of algorithms begins with the built-in models. You can add custom Python models at the bottom of the list. Click the edit button to change the model name, displayed in the output, from the default “Custom Python Model”. Dataiku DSS provides some code samples to get you started:
The code must follow some constraints depending on the backend you have chosen (in-memory or MLlib).
In this example, we use the Python in-memory backend: the custom code needs to implement a classifier which has the same methods as a classifier in scikit-learn, that is, it must provide the methods
predict_proba() when this makes sense.
Dataiku DSS will leverage these in its machine learning pipeline to train, compute performance statistics, and generate the associated visual insights!
Session Output With A Custom Model¶
Once trained, your custom model appears in the session output in the list of all models built during the session.
Assessing Custom Model performance¶
Open the custom model to visualize its performance and all associated visual insights, just as you would with a built-in model.
The custom model can now be deployed in the flow and used just like a standard built-in model!
Please read the reference documentation to implement your own MLlib models in Scala while still using Dataiku DSS modeling in the Visual Analysis.
If modeling in the Visual Analysis with custom code does not suit your needs, you can also take full control by coding the whole machine learning pipeline by yourself (train, score, validation, etc) using your preferred languages (python, R, Scala or Shell), thus leveraging any external ML libraries.