Thursday, December 12, 2019

SQL vs CSV vs PKL for data storage

The CSV files are sufficient to build a proof of concept. Given that sufficiency, and my lack of familiarity with SQL, perhaps I shouldn't focus on using sqlite for the MVP.
(See https://physicsderivationgraph.blogspot.com/2019/12/mvp-for-pdg-with-sql.html)

CSVs feel hacky and don't enable enforcement of consistency checks. However, they are easier to view and decrease the dependencies.

The easiest solution might be using Pickle (PKL) files rather than converting from Python variables in-memory to a distinct off-line representation. A Pickle file is enables consolidation.

No comments:

Post a Comment