Examples

This example illustrates:

• How to setup a Disdat project to use TensorFlow (require tensorflow in your setup.py)

• How to assemble Disdat tasks into a pipeline

• How task outputs become bundles

• How to include extra data in your Docker container

Setup

Like the Tutorial, let's clone our examples repo:

1. Clone our examples github repo (https://github.com/seanr15/disdat-examples) here.

2. We'll assume you've installed it in $CODE

3. cd into the $CODE.

Note that it has its own setup.py, defining the requirements for our example pipelines (in the pipelines package). Let's install those requirements into your virtual environment.

$ cd $CODE
$ pip install -e .

We assume you have made an examples context and have switched into it. If not:

$ dsdt context examples
$ dsdt switch examples

Optional

If you want to push/pull your MNIST data to/from S3, then we assume you have bound the local context examples to your remote (described here). So if you list your contexts, you should see that you are in examples and its bound to a directory s3://disdat-prod/ on AWS.

$dsdt context
*	examples	[examples@s3://disdat-prod/context]

Now on to TF and MNIST