moving from local terminal interface with CSV to web interface with a database backend

The existing Physics Derivation Graph is centered on command-line input with PNG output. Recently I've been exploring use of d3js as a web frontend.

I'm considering moving the Physics Derivation Graph to something which supports a web frontend connected to a hosted database backend. No command-line interaction or installation required.

I've been using CSV to store the data because it is the most common format. That works fine as long as the PDG is local and command-line driven. Now I want to move to a website, (ie openshift --
https://openshift.redhat.com/app/console/application_type/ )
I've set up a Django cartridge

I realized I don't need to sketch a design for the PDG web interface. Instead, I can start with determining minimum functionality:

• user login (don't want spammers writing to the database)
• user view existing content
• user add new content

I've used Flask previously, but I think the functionality I want is sufficiently complex that Flask wouldn't be enough.

I don't know whether I could continue to use the existing CSV as a backend storage. At this point I don't need scalability in terms of user latency or amount of data.


Vision: enable users of the PDG to interact with content in a web interface

Goal: use Django on Openshift to enable authentication, storage of content, and manipulating content

Path:

1. Flask local instance
• https://hplgit.github.io/web4sciapps/doc/pub/._web4sa_flask006.html
• http://code.tutsplus.com/tutorials/an-introduction-to-pythons-flask-framework--net-28822
2. Flask read values from database - CSV
3. convert PDG CSV to local SQL database
4. read from local SQL database
5. write to local SQL database - https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/sqlite3.1.html
6. Flask read values from database - SQL
7. Django local instance
8. Django local instance read values from database
9. Django local instance write values from database
10. Django on Openshift - http://physicsderivationgraph.blogspot.com/2015/11/django-and-openshift.html
11. Django on Openshift read values from database
12. Django on Openshift write values from database
13. Django on Openshift write values from database by authenticated user

Database choices

https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems
https://www.digitalocean.com/community/tutorials/a-comparison-of-nosql-database-management-systems-and-models

CSV

SQLite

SQLite only supports a single writer at a time (meaning the execution of an individual transaction). SQLite locks the entire database when it needs a lock (either read or write) and only one writer can hold a write lock at a time.
SQLite 3.7.0 added a new journal mode called Write Ahead Locking that supports concurrent reading while writing.