LogoLogo
  • Overview
  • Setup and Configuration
  • Other Data Versioning Systems
  • Examples
    • Tutorial
      • Creating Bundles with the Python API
      • Push/Pull using S3
      • Simple Pipeline
      • Run the Pipeline
      • Dockerize a Pipeline
      • Run the Pipeline Container (locally)
      • Run the Pipeline Container (AWS)
    • Examples
      • MNIST and TensorFlow
      • Spacy Task
  • Basic Concepts
    • Bundles
      • Naming
      • Bundle Data Types
      • Tags and Parameters
      • Lineage (or Bundle Metadata)
    • Data Contexts
  • Reference
    • CLI Reference
      • dsdt add
      • dsdt apply
      • dsdt cat
      • dsdt context
      • dsdt commit
      • dsdt dockerize
      • dsdt init
      • dsdt lineage
      • dsdt ls
      • dsdt pull
      • dsdt push
      • dsdt remote
      • dsdt rm
      • dsdt rmr
      • dsdt switch
    • Python API
  • Details
  • Building Pipelines
  • Running Pipelines on AWS
  • Admin
    • Contact / Slack
Powered by GitBook
On this page

Was this helpful?

  1. Basic Concepts
  2. Bundles

Bundle Data Types

That's a nice bundle you have there.

Bundle Data Presentation Types

Kind

Type

Example

literal or file link

int, float, str, bool, unicode

return 3 return 3.14 return "pi" return True

return "s3://abucket/akey/afile.parquet"

None type

NoneType

return None

flat collection of

literals and file links

list, tuple

return [1,3,5] return ("jd", "ke")

dictionary of flat

collections

dictionary

return {'name':["mark","sue"], 'age':[21, 32]}

file links

str or luigi.LocalTarget

return /Users/kyocum/fancy.csv

return s3://abucket/akey/afile.parquet

dataframe

pandas.DataFrame

return pd.DataFrame({'name':['mark','sue']})

Currently Disdat only supports simple dataframes (no hierarchical indices or complex object types).

If you want to store something complex, there are many good file formats that already exist: csv, parquet, orc, etc. Choose one of those and store the link!

PreviousNamingNextTags and Parameters

Last updated 5 years ago

Was this helpful?