Deploy (video)



Previously, we discussed how to define, train and evaluate machine learning models in DSS.

In this video, we’ll examine model deployment and the scoring of unseen data.

Specifically, we are going to cover three key concepts: 

  • deploying machine learning models to the flow, 
  • batch scoring of data using a Predict recipe,
  • and real time scoring of data using API nodes.

First, let’s discuss deploying models to the flow.

We start in the Visual Analysis from our previous video.

Now that we’ve finished experimenting, we would like to use one of our trained models to score unseen data.

To do this, we must first deploy the selected model to the flow.

Deploying a model to the flow generates a training recipe and a model object.

The training recipe enables models to be retrained directly from the flow as new data becomes available.

The model object, on the other hand, exposes the currently active model to other elements in the flow.

Additionally, the model object also contains a copy of all previously active models.

DSS makes it clear what version of the model is active at any given time, and makes it easy to fall back to a previously-deployed version when necessary.

Now, let’s discuss using deployed models to batch score unseen data.

Batch scoring involves collecting data over some period of time before passing it to a model to generate predictions.

In DSS, we can batch score a dataset using the Predict recipe.

For example, the kaggle_submission dataset contains information for almost 10,000 taxi rides.

We can use a Predict recipe, together with the model deployed to the flow, to batch score the taxi rides, and thus generate a fare prediction for each ride.

Batch scoring is useful for asynchronous applications, but how can we leverage DSS to provide near-instant predictions?

Enter real time scoring through API nodes!

API Nodes expose DSS objects to external applications through a REST API interface.

Amongst other objects, API Nodes can host services exposing machine learning models, python and r functions, SQL queries, and DSS datasets.

Now, let’s discuss creating a Prediction Model API service to enable real time scoring of data.

We begin with a machine learning model deployed to the flow.

To create an API Service, we send the model to the API Designer.

In addition to the exposed model, the API Designer allows us to define test queries, query enrichments, and the scoring engine.

We then push the API Service to the API Deployer.

As its name implies, the API Deployer manages the deployment of API Services to the API Nodes.

The API Deployer additionally manages the versioning of API Services.

For example, we might decide to create a new version of our API Service following an update to the underlying machine learning model.

We would then push this new version to the API Deployer, which now contains two versions of our API Service.

Finally, we can use the API Deployer to deploy a version of our service to an API node.

Note that the API Deployer is not bound to a specific project.

Instead, it can be used to deploy API services from any project on the DSS instance.

Let’s now apply what we’ve learnt to the NY Taxi Fares project.

We’ll create an API service called “NY Taxi Service”, with a “fare prediction” endpoint.

From the API Designer, we can now select the scoring engine add test queries from the “kaggle submission” dataset, and optionally include query enrichments.

Query enrichments are used to complement the information provided by client applications.

For example, a client application may request a prediction from an API endpoint.

In this case, two features are included in the model query, while the API model requires three.

To resolve this discrepancy, we enrich the query to add the third required feature.

The source of the enrichment can be either a bundled or a referenced database.

Finally, we score the enriched query using the API model and send a prediction response to the client application.

As we don’t require query enrichments for our taxi fares prediction, we can go ahead and push “version one” of the API service to the API Deployer.

As mentioned previously in this video, the API Deployer contains API Services from multiple DSS projects.

Navigating to “version one” of our Taxi Fares service, we can deploy it to an API Node.

DSS API Nodes can be installed either on a static server, or a scalable kubernetes cluster.

In this case, we’ll select a static development infrastructure to deploy our API service.

Once deployed, we’ll execute the test queries we added in the API Designer to ensure the service was correctly deployed.

Fantastic! It looks like our model has been successfully deployed to an API Node!

As a final step, let’s check out an external application that leverages our deployed model.

If we head over to we’ll find a simple webapp that predicts the cost of a taxi ride in New York City.

After entering some taxi ride details, the webapp will send a prediction request to a DSS API Node.

After a brief period of time, the DSS API Node sends a response, which in our case is predicting a cost of X dollars.

That concludes our introduction into model deployment and scoring.

Congratulations, and thank you for watching!