How to manage task and pipeline templates

Manage template files on Google Storage

Go to the bucket dedicated to the storage of template files. There you will find the "pipelines" and "tasks" folders. To ensure proper working order, these folders must not be renamed or modified.

cookbook16 1

However, everything in these folders is fully customizable, you can create as many sub-folders and custom files to create your custom categories and templates.

cookbook16 2

In the example below. I created two custom manifests that I called "my-job-fibinacci.json" and "my-job-prediction.json" that I inserted in the "my-custom-jobs" folder (visible above).

cookbook16 3

All files must have the json extension and contain a valid json format otherwise it may not be supported by Datatask.

Google Storage’s replicated structure

As you can see below, the structure of Google storage is reproduced. Folders are categories and files are templates.

cookbook16 4