Physics Derivation Graph: June 2018

Tuesday, June 19, 2018

why software used by the Physics Derivation Graph is open source

The Physics Derivation Graph content is available under the Creative Commons Attribution 4.0 International License, and I use open source software.

I avoid software that is not open source and not free

Matlab
Mathematica
Maple
Magma
MathCAD

My motivations for this include

wider accessibility of the results due to fewer constraints
enable other people to build on top of the results
contribute back to the community which has provided so much

community growth and diverging interests

Recently the number of developers involved in the Physics Derivation Graph has started to unexpectedly increase. As the number of people involved grows, the objectives diversify. People with shared interest can progress faster by collaborating.

There are multiple channels for people engaging in the project:

the github "group" has multiple repos, each a distinct but related effort
the gitter group has multiple discussions and one-to-one threads
email threads
phone and Skype calls

The idea of splitting the team was suggested in order to address the complexity of interaction. Having multiple threads going concurrently can be distracting to members. Then the question is how focused team members can be in the presence of distractions.

I usually let things like this go until there's a clear need for change. The necessity of the change usually helps push which option is preferable. Thinking ahead about potential change is useful, but the action can be delayed until needed.

Once the split occurs, what coordination among the groups is needed? How does it occur?

Monday, June 18, 2018

converting the old derivations into the new folder structure

Context: this post documents a one-time fix that converts old derivations to the new convention.

In the "version 4" implementation of the Physics Derivation Graph, there are two directories that contain Latex and PNG files of expressions and feeds:

in the directory of the derivation, ie "proofofconcept/v4_file_per_expression/derivations/frequency period relation"
in the directory containing all expressions, ie "proofofconcept/v4_file_per_expression/expressions"

One reason for this redundancy is to enable modification of the derivation content with disrupting the complete graph. Also, deconflicting expression index collisions doesn't need to be carried out until the derivation is verified. Lastly, I don't have an automatic method for deconflicting expression indices.

As of today (20180618), there are the derivations which were manually created don't follow the convention of having Latex and PNG in the folder of the specific derivation. These older derivations only have the Latex and PNG in the second directory.

In order to enable editing of existing derivations, I needed to copy expressions and feeds from the shared folder into each derivation. To do this, I started in a specific derivation folder and copied only the relevant LaTeX and PNG into the folder.

pwd
proofofconcept/v4_file_per_expression/derivations/derivation of Schrodinger Eq
while IFS='' read -r line || [[ -n "$line" ]]; do cp ../../feeds/${line}* .; done < feeds.csv
while IFS='' read -r line || [[ -n "$line" ]]; do cp ../../expressions/${line}* .; done < <(cat expression_identifiers.csv | cut -d',' -f2)

Saturday, June 2, 2018

building a docker image for the Physics Derivation Graph

Requirements

In order to build the Physics Derivation Graph in a Docker image, the minimum functionality needed is

Compile .tex files to .pdf and .png (LaTeX)
Compile .gv files to .png (GraphViz)
Run a webserver (eg flask or nginx or lighthttpd)
Run .py scripts (Python 3)
Read SQLite

In this post I explore whether Alpine is a sufficient OS. If not, Ubuntu is a candidate OS which supports the needed functionality.

Alpine-based

The build will be executed from the "proofofconcept" folder because the contents of the Docker image depend on files in the v4_file_per_expression/ folder.

cd proofofconcept
mkdir sandbox/docker_images/python_alpine
cat > sandbox/docker_images/python_alpine/Dockerfile << EOF

FROM python:2.7-alpine

MAINTAINER My Name <my.email.address@gmail.com>

LABEL distro_style="apk" distro="alpine" arch="x86_64" operatingsystem="linux"

RUN apk add --update --no-cache graphviz
RUN apk add --update --no-cache texlive-full
RUN apk add --update --no-cache texlive

RUN pip install pyyaml
RUN pip install sympy

RUN mkdir /derivations
RUN mkdir /inference_rules

ADD ./v4_file_per_expression/bin/interactive_user_prompt.py interactive_user_prompt.py
ADD ./v4_file_per_expression/lib/lib_physics_graph.py /lib/lib_physics_graph.py
ADD ./v4_file_per_expression/inference_rules/* /inference_rules/

#WORKDIR /bin

CMD ["python", "interactive_user_prompt.py"]

EOF

The above Dockerfile fails due to "texlive" and "texlive-full" not existing for alpine.

Now that the Dockerfile exists, we can build the image:
docker build --tag python_alpine/example sandbox/docker_images/python_alpine/
and run it to get the interactive prompt:
docker run -ti python_alpine/example

Ubuntu-based

cd proofofconcept
mkdir sandbox/docker_images/python_ubuntu

cat > sandbox/docker_images/python_ubuntu/Dockerfile << EOF

# 20180602

FROM ubuntu:18.04

MAINTAINER My Name <my.email.address@gmail.com>

RUN apt-get update \

&& apt-get upgrade -y \

&& apt-get install -y \

python-pip \

python2.7 \

graphviz

RUN pip install pyyaml

RUN pip install sympy

RUN mkdir /derivations

RUN mkdir /inference_rules

ADD ./v4_file_per_expression/bin/interactive_user_prompt.py interactive_user_prompt.py

ADD ./v4_file_per_expression/lib/lib_physics_graph.py /lib/lib_physics_graph.py

ADD ./v4_file_per_expression/inference_rules/* /inference_rules/

#WORKDIR /bin

#ENTRYPOINT ["/usr/bin/python2.7"]

CMD ["python", "interactive_user_prompt.py"]

EOF

docker build --tag python_ubuntu/example sandbox/docker_images/python_ubuntu/

Stop and remove all images

Not unexpectedly, I ran out of disk space.

docker stop $(docker ps -a -q)

docker rm $(docker ps -aq)

docker rmi -f $(docker images -q)