Bundle Data Types
That's a nice bundle you have there.
Bundle Data Presentation Types
Kind
Type
Example
literal or file link
int, float, str, bool, unicode
return 3
return 3.14
return "pi"
return True
return "s3://abucket/akey/afile.parquet"
None type
NoneType
return None
flat collection of
literals and file links
list, tuple
return [1,3,5]
return ("jd", "ke")
dictionary of flat
collections
dictionary
return {'name':["mark","sue"], 'age':[21, 32]}
file links
str or luigi.LocalTarget
return /Users/kyocum/fancy.csv
return s3://abucket/akey/afile.parquet
dataframe
pandas.DataFrame
return pd.DataFrame({'name':['mark','sue']})
Currently Disdat only supports simple dataframes (no hierarchical indices or complex object types).
If you want to store something complex, there are many good file formats that already exist: csv, parquet, orc, etc. Choose one of those and store the link!
Last updated