dsdt lineage
A history of a bundle came to be
Description
Every time you run a Disdat pipeline, Disdat tracks the code used to produce the data, the task's parameters, and the input bundles. This command gives you access to that information.
Usage
usage: dsdt lineage [-h] [-f] [-d DEPTH] uuid
Options
positional arguments:
uuid Find lineage by bundle UUID
optional arguments:
-h, --help show this help message and exit
-f, --follow Follow lineage back through D levels.
-d DEPTH, --depth DEPTH
Number of levels to follow (default is 3).
Examples
First, let's say we've run the following pipeline from our disdat-examples
repo.
$dsdt apply pipelines.dependent_tasks.B
...
===== Luigi Execution Summary =====
Scheduled 3 tasks of which:
* 3 ran successfully:
- 1 A(...)
- 1 B(...)
- 1 DriverTask(...)
This progress looks :) because there were no failed tasks or missing dependencies
===== Luigi Execution Summary =====
We can then look at the lineage of bundle B
$dsdt lineage 2871b1a4-1ba1-43f3-a8d8-9cc67b6f51f9
------ Lineage @ depth 0 -----
processing name: B__99914b932b_578baf4cdb
uuid: 2871b1a4-1ba1-43f3-a8d8-9cc67b6f51f9
creation date: 2020-06-10 00:01:32.851995
code repo: git@github.com:seanr15/disdat-examples.git
code hash: 60e511b6f077827d6269a82f87d1558a29513fc1
code method: pipelines.dependent_tasks.B
git commit URL: https://github.com/seanr15/disdat-examples/commit/60e511b6f077827d6269a82f87d1558a29513fc1
code branch: kyocum/file-examples
Start 1591772492.851795 Stop 1591772492.85182 Duration 2.5033950805664062e-05
------ Lineage @ depth 1 -----
processing name: A__99914b932b_d41d8cd98f
uuid: 8e1eeb9b-cd6a-49cc-b462-37deaa05a286
creation date: 2020-06-10 00:01:32.831018
code repo: git@github.com:seanr15/disdat-examples.git
code hash: 60e511b6f077827d6269a82f87d1558a29513fc1
code method: pipelines.dependent_tasks.A
git commit URL: https://github.com/seanr15/disdat-examples/commit/60e511b6f077827d6269a82f87d1558a29513fc1
code branch: kyocum/file-examples
Start 1591772492.829466 Stop 1591772492.829489 Duration 2.288818359375e-05
Last updated
Was this helpful?