dsdt lineage

A history of a bundle came to be

Description

Every time you run a Disdat pipeline, Disdat tracks the code used to produce the data, the task's parameters, and the input bundles. This command gives you access to that information.

Usage

usage: dsdt lineage [-h] [-f] [-d DEPTH] uuid

Options

positional arguments:
  uuid                  Find lineage by bundle UUID

optional arguments:
  -h, --help            show this help message and exit
  -f, --follow          Follow lineage back through D levels.
  -d DEPTH, --depth DEPTH
                        Number of levels to follow (default is 3).

Examples

First, let's say we've run the following pipeline from our disdat-examples repo. 

$dsdt apply pipelines.dependent_tasks.B
...
===== Luigi Execution Summary =====

Scheduled 3 tasks of which:
* 3 ran successfully:
    - 1 A(...)
    - 1 B(...)
    - 1 DriverTask(...)

This progress looks :) because there were no failed tasks or missing dependencies

===== Luigi Execution Summary =====

We can then look at the lineage of bundle B

$dsdt lineage 2871b1a4-1ba1-43f3-a8d8-9cc67b6f51f9
------ Lineage @ depth 0 -----
processing name: B__99914b932b_578baf4cdb
uuid: 2871b1a4-1ba1-43f3-a8d8-9cc67b6f51f9
creation date: 2020-06-10 00:01:32.851995
code repo: git@github.com:seanr15/disdat-examples.git
code hash: 60e511b6f077827d6269a82f87d1558a29513fc1
code method: pipelines.dependent_tasks.B
git commit URL: https://github.com/seanr15/disdat-examples/commit/60e511b6f077827d6269a82f87d1558a29513fc1
code branch: kyocum/file-examples
Start 1591772492.851795 Stop 1591772492.85182 Duration 2.5033950805664062e-05

	------ Lineage @ depth 1 -----
	processing name: A__99914b932b_d41d8cd98f
	uuid: 8e1eeb9b-cd6a-49cc-b462-37deaa05a286
	creation date: 2020-06-10 00:01:32.831018
	code repo: git@github.com:seanr15/disdat-examples.git
	code hash: 60e511b6f077827d6269a82f87d1558a29513fc1
	code method: pipelines.dependent_tasks.A
	git commit URL: https://github.com/seanr15/disdat-examples/commit/60e511b6f077827d6269a82f87d1558a29513fc1
	code branch: kyocum/file-examples
	Start 1591772492.829466 Stop 1591772492.829489 Duration 2.288818359375e-05
Previousdsdt initNextdsdt ls

Last updated