why software used by the Physics Derivation Graph is open source

The Physics Derivation Graph content is available under the Creative Commons Attribution 4.0 International License, and I use open source software.

• Python license
• LaTeX
• graphviz license
• OCaml license
• SymPy license
• Sage

I avoid software that is not open source and not free

• Matlab
• Mathematica
• Maple
• Magma
• MathCAD

My motivations for this include

• wider accessibility of the results due to fewer constraints
• enable other people to build on top of the results
• contribute back to the community which has provided so much 

community growth and diverging interests

Recently the number of developers involved in the Physics Derivation Graph has started to unexpectedly increase. As the number of people involved grows, the objectives diversify. People with shared interest can progress faster by collaborating.

There are multiple channels for people engaging in the project:

• the github "group" has multiple repos, each a distinct but related effort
• the gitter group has multiple discussions and one-to-one threads
• email threads
• phone and Skype calls

The idea of splitting the team was suggested in order to address the complexity of interaction. Having multiple threads going concurrently can be distracting to members. Then the question is how focused team members can be in the presence of distractions.

I usually let things like this go until there's a clear need for change. The necessity of the change usually helps push which option is preferable. Thinking ahead about potential change is useful, but the action can be delayed until needed. 

Once the split occurs, what coordination among the groups is needed? How does it occur?

converting the old derivations into the new folder structure

Context: this post documents a one-time fix that converts old derivations to the new convention.

In the "version 4" implementation of the Physics Derivation Graph, there are two directories that contain Latex and PNG files of expressions and feeds:

• in the directory of the derivation, ie "proofofconcept/v4_file_per_expression/derivations/frequency period relation"
• in the directory containing all expressions, ie "proofofconcept/v4_file_per_expression/expressions"

One reason for this redundancy is to enable modification of the derivation content with disrupting the complete graph. Also, deconflicting expression index collisions doesn't need to be carried out until the derivation is verified. Lastly, I don't have an automatic method for deconflicting expression indices.

As of today (20180618), there are the derivations which were manually created don't follow the convention of having Latex and PNG in the folder of the specific derivation. These older derivations only have the Latex and PNG in the second directory.

In order to enable editing of existing derivations, I needed to copy expressions and feeds from the shared folder into each derivation. To do this, I started in a specific derivation folder and copied only the relevant LaTeX and PNG into the folder.

pwd
proofofconcept/v4_file_per_expression/derivations/derivation of Schrodinger Eq
while IFS='' read -r line || [[ -n "$line" ]]; do cp ../../feeds/${line}* .; done < feeds.csv
while IFS='' read -r line || [[ -n "$line" ]]; do cp ../../expressions/${line}* .; done < <(cat expression_identifiers.csv | cut -d',' -f2)

building a docker image for the Physics Derivation Graph

Requirements

In order to build the Physics Derivation Graph in a Docker image, the minimum functionality needed is

• Compile .tex files to .pdf and .png (LaTeX)
• Compile .gv files to .png (GraphViz)
• Run a webserver (eg flask or nginx or lighthttpd)
• Run .py scripts (Python 3)
• Read SQLite

In this post I explore whether Alpine is a sufficient OS. If not, Ubuntu is a candidate OS which supports the needed functionality.

Alpine-based

The build will be executed from the "proofofconcept" folder because the contents of the Docker image depend on files in the v4_file_per_expression/ folder.

cd proofofconcept
mkdir sandbox/docker_images/python_alpine
cat > sandbox/docker_images/python_alpine/Dockerfile << EOF

FROM python:2.7-alpine

MAINTAINER My Name <my.email.address@gmail.com>

LABEL distro_style="apk" distro="alpine" arch="x86_64" operatingsystem="linux"

RUN apk add --update --no-cache graphviz
RUN apk add --update --no-cache texlive-full
RUN apk add --update --no-cache texlive

RUN pip install pyyaml
RUN pip install sympy

RUN mkdir /derivations
RUN mkdir /inference_rules

ADD ./v4_file_per_expression/bin/interactive_user_prompt.py interactive_user_prompt.py
ADD ./v4_file_per_expression/lib/lib_physics_graph.py /lib/lib_physics_graph.py
ADD ./v4_file_per_expression/inference_rules/* /inference_rules/

#WORKDIR /bin

CMD ["python", "interactive_user_prompt.py"]

EOF

The above Dockerfile fails due to "texlive" and "texlive-full" not existing for alpine.

Now that the Dockerfile exists, we can build the image:
docker build --tag python_alpine/example sandbox/docker_images/python_alpine/
and run it to get the interactive prompt:
docker run -ti python_alpine/example

Ubuntu-based

cd proofofconcept
mkdir sandbox/docker_images/python_ubuntu

cat > sandbox/docker_images/python_ubuntu/Dockerfile << EOF

# 20180602

FROM ubuntu:18.04

MAINTAINER My Name <my.email.address@gmail.com>

RUN apt-get update \

    && apt-get upgrade -y \

    && apt-get install -y \

    python-pip \

    python2.7 \

    graphviz    

RUN pip install pyyaml

RUN pip install sympy

RUN mkdir /derivations

RUN mkdir /inference_rules

ADD ./v4_file_per_expression/bin/interactive_user_prompt.py interactive_user_prompt.py

ADD ./v4_file_per_expression/lib/lib_physics_graph.py /lib/lib_physics_graph.py

ADD ./v4_file_per_expression/inference_rules/* /inference_rules/

#WORKDIR /bin

#ENTRYPOINT ["/usr/bin/python2.7"]

CMD ["python", "interactive_user_prompt.py"]

EOF

docker build --tag python_ubuntu/example sandbox/docker_images/python_ubuntu/

Stop and remove all images

Not unexpectedly, I ran out of disk space.

docker stop $(docker ps -a -q)

docker rm $(docker ps -aq)

docker rmi -f $(docker images -q)

lots of channels for tracking exploration

A private Trello board for task tracking:
https://trello.com/b/kSZvdVg5/physics-derivation-graph

Primary source code repo:
https://github.com/allofphysicsgraph/proofofconcept/blob/gh-pages/doc/physics_graph_notes.log

Demo of proof of concept:
http://allofphysicsgraph.github.io/proofofconcept/

A blog to track my status
https://physicsderivationgraph.blogspot.com/

An old set of descriptions
https://sites.google.com/site/physicsderivationgraph/
This site is depreciated

limiting my effort to just Latex for expressions

The core of the Physics Derivation graph is the relation between expressions. I've considered how much additional knowledge could be captured by storing expressions as Abstract Syntax Trees. While this would add a lot of work and therefore take more time, there's potentially a lot of value in having a more robust representation. Additionally, there's the challenge that I don't know how to represent all expressions in Physics using Abstract Syntax Trees.

Rather than store each expression as an AST, I'm going to limit my effort to just storing expressions using Latex. As a consequence, the validity of inference rules applied to expressions not be able to be checked. If someone comes up with a representation more useful than Latex (ie ASTs, MathML, etc), then a conversion will need to be performed.

The second reason the choice of Latex is significant because it limits how far down in the hierarchy that can be enumerated. Specifically, in the context of these layers

• Physics derivation graph
• Derivation 
• Step 
• Expressions, inference rules 
• Symbol, operators

the Physics Derivation Graph with Latex will not be able to explore systematically the symbols and operators used.

validating equivalent and transformed expressions

To check whether a step was implemented correctly, the question is, "given the input expressions (and feeds), do the output expressions conform to the transform prescribed by the inference rule?"

For example, suppose we start with "A=B" and operate on this with the inference rule "add X to both sides," where X=2. If the output is "A+2 = B+2," was the step implemented correctly?

The inference rule can be described by an abstract syntax tree (AST):

    =                      =
  /   \        -->      /     \
LHS   RHS              +        +
                     /   \    /   \ 
                   LHS   x   RHS   x


https://blog.plover.com/math/24-puzzle-2.html
https://news.ycombinator.com/item?id=15075110