The Physics Derivation Graph depends on a data structure capable of using ordered lists. RDF's support for ordered lists is slightly convoluted. The best visualization of ordered lists in RDF I've found is https://ontola.io/blog/ordered-data-in-rdf/
I tried sketching how the "linked recursive lists" approach looks for the Physics Derivation Graph for a derivation that has a sequence of steps, and each step has an ordered list of inputs, feeds, and outputs.
Credit: dreampuf.github.io
Classes and subclasses of entities in the Physics Derivation Graph:
I've wrestled with whether to use Property Graphs to store and query the Physics Derivation Graph. I see potential value, but the licensing of Neo4j keeps me from committing. I'm aware of other implementations, but I don't have confidence about either their stability or durability.
This post makes a convincing argument about both the short-comings of a property-graph-based knowledge graph and the value of an RDF-based storage method. To summarize,
I know OWL (Web Ontology Language) is popular for knowledge representation, and this post was the first to provide a clear breakdown of the difference between property graphs, RDF, and OWL. OWL supports
My first step is to review logins on the site,
https://physicsderivationgraph.blogspot.com/2020/05/inspecting-list-of-users-who-have.html
My previous post on reviewing logs
https://physicsderivationgraph.blogspot.com/2020/05/grepping-nginx-logs-to-observe-user.html
was written prior to the current nginx format I'm using.
I haven't gotten around to a deeper analysis like
https://physicsderivationgraph.blogspot.com/2020/04/analysis-of-web-logs-to-understand-how.html
First I had to install supporting software
sudo apt install python3-pip pip3 install pandas
Inline Python in bash with Pandas is possible because every line is formatted like a Python dictionary. Here I want to review what columns are present in the logs
cat nginx_access.log | python3 -c "import sys
import pandas
pandas.options.display.max_rows = 999 # https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html
list_of_lines = []
for line in sys.stdin:
list_of_lines.append(eval(line))
df = pandas.DataFrame(list_of_lines)
print(df.columns)
"
How many of each entry for a few columns?
cat nginx_access.log | python3 -c "import sys
import pandas
pandas.options.display.max_rows = 999 # https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html
list_of_lines = []
for line in sys.stdin:
list_of_lines.append(eval(line))
df = pandas.DataFrame(list_of_lines)
threshold = 20
print('user:')
vc = df['user'].value_counts()
print(vc[vc>threshold])
print('IP:')
vc = df['ip'].value_counts()
print(vc[vc>threshold])
print('req:')
vc = df['req'].value_counts()
print(vc[vc>threshold])
#print(df.head())
"
For IPs that have made multiple (e.g., 30) requests, what pages have been accessed?
cat nginx_access.log | python3 -c "import sys
import pandas
pandas.options.display.max_rows = 999 # https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html
list_of_lines = []
for line in sys.stdin:
list_of_lines.append(eval(line))
df = pandas.DataFrame(list_of_lines)
threshold = 30
vc = df['ip'].value_counts()
for ip, number_of_requests in vc[vc>threshold].items():
print('\nIP = ',ip, 'made',number_of_requests,'requests')
df_this_ip = df[df['ip']==ip]
#for request in df_this_ip['req'].values:
# print(request)
print(df_this_ip['req'].value_counts())
"
I've been running a DigitalOcean droplet for $5/month for the past 6 months. Because I was new and didn't know better, I selected the Ubuntu 18.04 droplet.
Now I want to update to Ubuntu 20.04 LTS.
The guide recommends starting with a fresh 20.04 image instead of upgrading.
The following is a record of the steps I took in this process.
Total duration: 2 hours. The process took longer than expected because I hadn't previously configured the website from a bare Ubuntu server. Also, I had made a few changes since the initial installation that weren't documented.
Used scp to copy data from the droplet to my mac
scp user@IP:/home/pdg/arxiv_rss/rss_filter_email.py .
scp user@IP:/home/pdg/arxiv_rss/.env .
scp user@IP:/home/pdg/videos/* .
scp user@IP:/home/pdg/.bash_history .
scp user@IP:/home/pdg/.bashrc .
scp user@IP:/home/pdg/.python_history .
scp user@IP:/home/pdg/.sqlite_history .
cd proofofconcept/v7_pickle_web_interface/
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/.env .
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/certs/* .
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/flask/logs/* .
scp user@IP:/home/pdg/.ssh/authorized_keys .
Grab the crontab entry
0 0 * * * /usr/bin/python3 /home/user/arxiv_rss/rss_filter_email.py >> /home/user/arxiv_rss/cron.log 2>&1
https://www.digitalocean.com/docs/images/snapshots/how-to/snapshot-droplets/
Instead of expecting the user to provide the inputs and outputs and inference rule, supplying the inputs and inference rule is sufficient to generate the output. This output is necessarily consistent with the inputs and inference rule.
>>> from sympy import *
Define an inference rule
def mult_both_sides_by(expr, feed):
return Equality(expr.lhs*feed, expr.rhs*feed, evaluate=False)
>>> expr = parse_latex('a = b')
>>> feed = parse_latex('f')
>>> mult_both_sides_by(expr, feed)
Eq(a*f, b*f)
This generalizes to include the relation
def mult_both_sides_by(expr, feed, relation):
return relation(expr.lhs*feed, expr.rhs*feed, evaluate=False)
>>> mult_both_sides_by(expr, feed, Equality)
Eq(a*f, b*f)
Other relations are available; see https://docs.sympy.org/latest/modules/core.html
>>> mult_both_sides_by(expr, feed, Le)
a*f <= b*f
As an illustration of the gradations from text to Latex to CAS is provided below. In the derivation the CAS is 1-to-1 with the Latex.
statement
Frequency and period are inversely related.
use of a Computer algebra system to implement inference rules
The following expansion requires
>>> import sympy
>>> from sympy import *
>>> from sympy.parsing.latex import parse_latex
>>> latex(eval(sympy.srepr(parse_latex('T = 1/f'))))
'T = \\frac{1}{f}'
We'll work with the SymPy representation of expression 1,
>>> sympy.srepr(parse_latex('T = 1/f'))
"Equality(Symbol('T'), Pow(Symbol('f'), Integer(-1)))"
Rather than using the SymPy, use the raw format of expression 1
>>> expr1 = parse_latex('T = 1/f')
inference rule: Multiple both sides of expression 1 by f to get expression 2
expression 2: f T=1
Although we can multiply a variable and an expression,
>>> expr1*Symbol('f')
f*(Eq(T, 1/f))
what actually needs to happen is first split the expression, then apply the multiplication to both sides
>>> Equality(expr1.lhs*Symbol('f'), expr1.rhs*Symbol('f'))
Eq(T*f, 1)
Application of an inference rule (above) results in the desired result, so save that result as the second expression (below).
>>> expr2 = Equality(expr1.lhs*Symbol('f'), expr1.rhs*Symbol('f'))
inference rule: divide both sides of expression 2 by T to get expression 3
expression 3: f = 1/T.
>>> Equality(expr2.lhs/Symbol('T'), expr2.rhs/Symbol('T'))
Eq(f, 1/T)
Again, save that to a variable
>>> expr3 = Equality(expr2.lhs/Symbol('T'), expr2.rhs/Symbol('T'))
>>> latex(expr3)
'f = \\frac{1}{T}'
inference rule: declare final expression
statement with inference rules and numeric IDs for symbols
To relate the above derivation to any other content in the Physics Derivation Graph, replace T and f with numeric IDs unique to "period" and "frequency"
>>> import sympy
>>> from sympy import *
>>> from sympy.parsing.latex import parse_latex
inference rule: Multiple both sides of expression 1 by f to get expression 2
expression 2: f T=1
removal of text, pure Python
The above steps can be expressed as a Python script with two functions (one for each inference rule)
from sympy import *
from sympy.parsing.latex import parse_latex
# assumptions: the inference rules are correct, the conversion of symbols-to-IDs is correct, the Latex-to-SymPy parsing is correct
def mult_both_sides_by(expr, feed):
return Equality(expr.lhs*feed, expr.rhs*feed)
def divide_both_sides_by(expr, feed):
return Equality(expr.lhs/feed, expr.rhs/feed)
# inference rule: declare initial expression
expr1 = parse_latex('T = 1/f')
expr1 = eval(srepr(expr1).replace('T','pdg9491').replace('f','pdg4201'))
feed = Symbol('f')
feed = eval(srepr(feed).replace('f','pdg4201'))
expr2 = mult_both_sides_by(expr1, feed)
feed = Symbol('T')
feed = eval(srepr(feed).replace('T','pdg9491'))
expr3 = divide_both_sides_by(expr2, feed)
latex(eval(
srepr(expr3).replace('pdg9491','T').replace('pdg4201','f')))
# inference rule: declare final expression
How would the rigor of the above be increased?
To get beyond what a CAS can verify, a "proof" would relate each of the two functions to a set of axioms. Given the two arguments (an expression, a "feed" value), is the returned value always consistent with some set of axioms?
The set of axioms chosen matters. For example, we could start with Zermelo–Fraenkel set theory
That would leave a significant gap between building up addition and subtraction and getting to calculus and differential equations. "Theorems of calculus derive from the axioms of the real, rational, integer, and natural number systems, as well as set theory." (source)
Here is what I currently understand to be true:
The consequence of the CAS-based graph not having 1-to1 correspondence with a Latex-based graph is that the current data structure of Latex and SymPy in one graph is not suitable.
I encountered a few issues with SymPy today. I'll explain what I found and what I plan to do about it.
I have an expression (1158485859) that, in Latex, is
\frac{-\hbar^2}{2m} \nabla^2 = {\cal H}
The \nabla^2 is represented in SymPy as Laplacian(), though an argument is required for the operator.
My solution: leave the SymPy representation as Pow(Symbol('nabla'), Integer(2))
Also on the topic of vectors, I encountered the expression
\vec{p} \cdot \vec{F}(\vec{r}, t) = a
In order to convert that to SymPy, I'd need to specify a basis for `\vec{p}` (e.g., `from sympy.vector import CoordSys3D`). For example,
>>> N = CoordSys3D('N')
>>> p = Symbol('p_x')*N.i + Symbol('p_y')*N.j + Symbol('p_z')*N.k
>>> F = Symbol('F_x')*N.i + Symbol('F_y')*N.j + Symbol('F_z')*N.k
>>> p.dot(F)
F_x*p_x + F_y*p_y + F_z*p_z
However, that doesn't seem to extend to functions:
>>> p.dot(Function(F)(Symbol('r'), Symbol('t')))
Traceback (most recent call last):
...
TypeError: expecting string or Symbol for name
My solution: leave the SymPy representation as incorrect, using "multiplication" instead of "dot"
Converting "\vec{\psi}(r, t)" into SymPy is feasible
Function('vecpsi')(Symbol('r'), Symbol('t'))
but I can't figure out how to apply the dot product with a vector:
>>> import sympy
>>> from sympy import *
>>> from sympy.vector import CoordSys3D, Del, curl, divergence, gradient
>>> Symbol('vecp').dot( Function('vecpsi')(Symbol('r'), Symbol('t')) )
Traceback (most recent call last):
AttributeError: 'Symbol' object has no attribute 'dot'
This morning I was greeted with this warning from Chrome when visiting https://derivationmap.net
$ openssl x509 -dates -noout < /home/pdg/proofofconcept/v7_pickle_web_interface/certs/fullchain.pem notBefore=May 11 15:26:19 2020 GMT notAfter=Aug 9 15:26:19 2020 GMT
sudo certbot certonly --webroot \ -w /home/pdg/proofofconcept/v7_pickle_web_interface/certs \ --server https://acme-v02.api.letsencrypt.org/directory \ -d derivationmap.net -d www.derivationmap.net
sudo rm -rf /etc/letsencrypt/{live,renewal,archive}/{derivationmap.net,derivationmap.net.conf}/
Request new certs
sudo certbot certonly --webroot \ -w /home/pdg/proofofconcept/v7_pickle_web_interface/certs \ --server https://acme-v02.api.letsencrypt.org/directory \ -d derivationmap.net -d www.derivationmap.net
Copy new certs to directory that nginx mounts in Docker-compose
cd /home/pdg/proofofconcept/v7_pickle_web_interface/certs sudo cp /etc/letsencrypt/live/derivationmap.net/fullchain.pem . sudo cp /etc/letsencrypt/live/derivationmap.net/privkey.pem . sudo chown pdg:pdg privkey.pem openssl dhparam -out dhparam.pem 2048
Restart Docker-compose
docker-compose up --build --force-recreate --remove-orphans --detach
If the docker containers are not restarted, the changes made to the file on the host won't take effect.
Verify in a browser that https://derivationmap.net/ has the updated certificate.
Set a calendar reminder to renew the certificate