graph visualization isn't a requirement for the Physics Derivation Graph

The central concept of the Physics Derivation Graph is that there are mathematical links between expressions. This means that navigating the content can be intuitively done using a graph. However, navigating a graph visually doesn't scale well.

The Physics Derivation Graph doesn't have to be presented as a graph. A derivation is typically presented in a linear format.

Storing the content of the Physics Derivation Graph does not require a graph storage database.

technology the Physics Derivation Graph builds on

The Physics Derivation Graph builds on prior technologies. In practice, the need was identified, created from scratch, then found to have been already done similarly by other people.

EquationMap

Gellish

Godel numbering

build a link graph

The Physics Derivation Graph (PDG) site uses relative links. This complicates creating a graph of the site links using wget since -k can be used together with -O only if outputting to a regular file.

$ root_page=allofphysicsgraph.github.io/proofofconcept/

$ wget http://$root_page -q -O -  | grep -i -o '<a[^>]\+href[ ]*=[ \t]*"[^"]\+"' | sed 's/<a href="//' | sed 's/"//' | grep -v https | sort | uniq > list_of_pages

$ while read -r line; do this_page="$line"; wget $root_page$this_page -q -O - | grep -i -o '<a[^>]\+href[ ]*=[ \t]*"[^"]\+"' | sed 's/<a href="//' | sed 's/"//' | grep -v https | sort | uniq >> list_of_pages2; done < list_of_pages 

$ cat list_of_pages2 | grep -v http | sed 's/^/site\//' >> list_of_pages

$ cat list_of_pages | sort | uniq > list_of_pages_master

$ rm list_of_pages list_of_pages2

$ while read -r line; do this_page="$line"; wget $root_page$this_page -q -O - | grep -i -o '<a[^>]\+href[ ]*=[ \t]*"[^"]\+"' | sed 's/<a href="//' | sed 's/"//' | awk -v thispage="$this_page" '{print thispage" ->", $0";"}' >> graph_level2.gv; done < list_of_pages_master

Python code validation

https://sourcegraph.com

https://github.com/yinwang0/pysonar2
https://yinwang0.wordpress.com/2010/09/12/pysonar/

http://pychecker.sourceforge.net/
not that useful for working code:

$ python bin/generate_new_random_index.py 
expression permanent index: 3135868900
expression temporary index: 3901417
feed temporary index      : 4031120
inf rule temporary index  : 2688703

$ pychecker bin/generate_new_random_index.py 
Processing module generate_new_random_index (bin/generate_new_random_index.py)...
expression permanent index: 1210104125
expression temporary index: 7904139
feed temporary index      : 7185822
inf rule temporary index  : 2695903

Warnings...

bin/generate_new_random_index.py:15: Imported module (random) not used

detecting crackpots

Finding derivations which relate famous equations can lead to paths worn by crackpots. Well known equations like F=m*a and E=m*c^2 serve as good targets for non-experts. With sufficient math, laypersons have a difficult time distinguishing crackpots from legitimate experts.

Joseph A. Rybczyk

I was looking for the relation between E=m*c^2 and F=m*a

I found this page

http://www.mrelativity.net/relationshipef/the_relationship_between_e_and_f_p1.htm

Clue number 1: a private domain name -- mrelativity.net

Clue number 2: other papers attacking relativity

Clue number 3: lack of Latex

Clue number 4: unfamiliar jargon -- "Internal Energy of a Particle" 

visualize success and then deliver the minimum viable product

Today I applied two principles and made tremendous progress. First, I thought about what success meant for the Physics Derivation Graph. The central claim is that mathematical physics can be represented as a single graph. My insight was that I should simply focus on that. The second principle was determining the minimum viable product to achieve that vision of success.



The outcome is a hyperlinked SVG of concepts and topics:

http://allofphysicsgraph.github.io/proofofconcept/site/sandbox/topic_and_concept_graph.svg

This is a map of topics (ie Quantum Mechanics) and concepts (ie Schrodinger's Equation) in Physics.

I've drawn edges between concepts where there exists a paper detailing the mathematical connection

In the SVG, edges which are bold can be clicked to see the referenced URL


There's an equivalent PNG, but the hyperlinks aren't active:
http://allofphysicsgraph.github.io/proofofconcept/site/sandbox/topic_and_concept_graph.png


The point of this graph is that

1. the major topics in Physics are present
2. the major topics are linked to associated concepts
3. there are mathematical links between the concepts

The remaining work is to
4) fill in with additional concepts so that there is a path from any blue box (topic) to any other blue box through only red ellipses (concepts). Then the claim will be validated in theory. 
5) After that, go back and fill in the actual math using expressions and inference rules. 

6) Verify correctness of steps using a CAS

• break Latex expressions into constituent symbols and operators (ref 1, ref 2, ref 3)

Then I'm done!


The PDG isn't "complete" in accounting for all of Physics, but I didn't expect to go that far. Showing complete coverage of topics is sufficient.

What I want to do with the Physics Derivation Graph

For the past few years, completing the Physics Derivation Graph seemed infeasible in one lifetime. This meant I would get a partial result in one lifetime, or I'd have to find motivated collaborators willing to spend part of their lifetime. Working alone towards a partial result wasn't attractive, and I haven't had much success finding collaborators. Faced with these two options, I did almost no work on the project.

A few months ago I realized an exit strategy would be to get the Physics Derivation Graph to the point that I'd be comfortable handing it off to other folks. That way the project wouldn't be bottlenecked by my productivity. This insight resulted in focusing on bugs and inconsistencies which would need to be addressed prior to handing the project off to someone else.

Today I envisioned what success would look like in light of the central claim that "mathematical physics can be represented by a single graph."

• By "graph" I actually mean three levels of granularity: the symbols composing expressions as abstract syntax trees, expressions and inference rules composing derivations, and the relation between topics composed from derivations.
• The scope of Mathematical Physics is broad, but there are major topics which need to be included in order to claim completeness: quantum mechanics, classical mechanics, electromagnetics, thermodynamics, relativity, particle physics, cosmology, and astronomy. 

For each of the major topics, there are corresponding concepts that can be represented mathematically:

• quantum mechanics: particle in a box, Schrodinger's equation, uncertainty
• classical mechanics: work, F=m*a, momentum
• electromagnetics: Maxwell's equations
• thermodynamics: Planck's Law
• relativity: Dirac equation
• particle physics: Dirac equation
• cosmology
• astronomy

Finally, the claim is that each of these topics is related via mathematics. Example: harmonic oscillator occurs in quantum and classical

addressing the rendering versus verifiability challenge

For the Physics Derivation Graph, the primary output is visual. Thus rendering expressions should be beautiful. Latex is a natural choice and the entry is intuitive and straightforward.

In addition to rendering, the other task is to verify that the content is correct. This means using a computer algebra system (ie Mathematica, Octave). Latex is not amenable to CAS input because Latex can be mathematically ambiguous -- resolution depends on context.

One way to resolve this would be to stick with Latex, then convert to a CAS format for verifying correctness.

Sage

As an example, suppose I want to check that the expression "multbothsidesby" was correctly entered for input T/f=1, output T=f, with feed f. The Sage syntax looks like

T,f=var('T,f')
input_expr = T/f==1
expected_output_expr= T==f
expected_output_expr == input_expr*f

The above Sage returns true, building confidence that the step is valid. More simply,

T,f=var('T,f')
(T==f) == ((T/f==1)*f)

If Latex is to be used as the input, then we need to convert it to Sage syntax.

• declare each variable in Sage
• replace "=" in Latex with "=="
• convert the inference rule to something that can be checked

In addition to using a Sage notebook (https://cloud.sagemath.com), there's a one-time eval option -- http://sagecell.sagemath.org/
Calling a local installation of Sage is possible from Python, see http://ask.sagemath.org/question/8215/using-sage-in-a-python-cgi-script/

change of mindset to improve user experience

Instead of treating this software project from the perspective of a developer, approach it from that of a customer.

First customer action: download a zip. The zip contains scripts, databases, documentation
--> action for me: write a tool which creates a zip

Second customer action: run the contents of the zip
--> action for me: document dependencies
--> task: define what "working" means

I'll define "working" by the following use cases:

• for a single derivation, generate the PNG of the graph
• for each of N derivations, generate a PNG per derivation
• for all derivations, generate a PNG
• for a single derivation, generate the HTML/d3js of the graph
• for each of N derivations, generate a HTML/d3js per derivation
• for all derivations, generate a HTML/d3js
• user should be able to enter a new derivation

These tasks require subtasks

• generate PNG per expression
• generate new random index that doesn't conflict

Other actions:

--> publish inference rules to website

--> provide example for each inference rule

--> publish reports (currently PDF) as HTML

challenge tasks to be addressed within the Physics Derivation Graph

There are a set of challenge tasks for the PDG. Each task is addressed by one or more scripts

Below the scripts in
https://github.com/allofphysicsgraph/proofofconcept/tree/gh-pages/sandbox/file_per_expression/bin
are referenced

tasks associated with generating content:

• given a set of latex expressions, generate associated PNGs for feeds

create_picture_per_feed_expression.py

• given a set of latex expressions, generate associated PNGs for expressions

create_picture_per_feed_expression.py

• given a set of text, generate associated PNGs for inference rules (needed for derivation graph HTML?)
• find new uniq random index for feeds
• find new uniq random index (temp) for expression
• find new uniq random index (perm) for expression

viz of content

• from an existing set of connections for a single derivation, generate a static png of the graph

python create_picture_per_derivation.py "../derivations/name of derivation"

• from an existing set of connections for a two or more derivations, generate a static png of each graph

bash create_picture_for_each_derivation.sh

• from an existing set of connections for a two or more derivations, generate a static png of full graph

bash create_all_derivations.sh; bash create_picture_for_each_derivation.sh
or
bash create_all_derivations.sh; python create_picture_per_derivation.py "../derivations/all"

HTML viz of content

• from an existing set of connections for a single derivation, generate an HTML presentation of the graph
• from an existing set of connections for a two or more derivations, generate an HTML presentation of each graph
• from an existing set of connections for a two or more derivations, generate an HTML presentation of full graph

entry of content

• use terminal CLI to enter new derivation
• use web page to construct new derivation

CAS

• validate a single step of a derivation
• validate two consecutive steps in a derivation
• validate all steps in a single derivation
• validate all steps in all derivations