Sunday, August 16, 2020

how to edit the SymPy Latex parser and rebuild the antlr artifacts for a pull request

https://github.com/sympy/sympy/wiki/Development-workflow#fork-sympy-project

Go to https://github.com/sympy/sympy
Fork to https://github.com/bhpayne/sympy/
In https://github.com/bhpayne/sympy/ create a new branch, e.g. "floor-patch"
In the bhpayne/sympy:floor-patch branch, change three files


Then, in a local directory run
mkdir build_sympy
cd build_sympy
git clone https://github.com/bhpayne/sympy/

I use a Docker container to build SymPy
cat <<EOF >> Dockerfile
FROM phusion/baseimage:0.11

RUN apt-get update && \
    apt-get install -y \
               vim \
               python3 python3-pip python3-dev \
               wget \
               default-jre \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /usr/local/lib
RUN curl -O https://www.antlr.org/download/antlr-4.7.2-complete.jar
COPY sympy/ /opt/
RUN echo "alias python=python3" > /root/.bashrc
RUN ln -s /usr/bin/python3.6 /usr/bin/python
# import the pip package for integration of grammar with Python    
RUN pip3 install antlr4-python3-runtime mpmath
# build antlr grammar
WORKDIR /opt/sympy/parsing/latex
ENV CLASSPATH=".:/usr/local/lib/antlr-4.7.2-complete.jar:$CLASSPATH"
RUN java -jar /usr/local/lib/antlr-4.7.2-complete.jar LaTeX.g4 -no-visitor -no-listener -o _antlr
# from msgoff
COPY rename.py /opt/sympy/parsing/latex
RUN python3 rename.py
# set up Sympy
WORKDIR /opt/
RUN python3 setup.py install
EOF

A second file, created by msgoff, is used for the Antlr build process
cat <<EOF >> rename.py
import glob
import os
output_dir = "_antlr"
for path in glob.glob(os.path.join(output_dir, "LaTeX*.*")) + glob.glob(
    os.path.join(output_dir, "latex*.*")):
    offset = 0
    new_path = os.path.join(output_dir, os.path.basename(path).lower())
    with open(path, "r") as f:
        lines = [line.rstrip() + "\n" for line in f.readlines()]
    os.unlink(path)
    with open(new_path, "w") as out_file:
        if path.endswith(".py"):
            offset = 2
            out_file.write(header)
        out_file.writelines(lines[offset:])
EOF

Inside the container,
cd /scratch/sympy/sympy/parsing/latex
java -jar /usr/local/lib/antlr-4.7.2-complete.jar LaTeX.g4 -no-visitor -no-listener -o _antlr
python rename.py

Now rebuild sympy
cd /scratch/sympy/
python setup.py install

leave the container
exit

On the host, add the build artifacts for Antlr
cd sympy/
git status
git add sympy/parsing/latex/_antlr/latexlexer.py sympy/parsing/latex/_antlr/latexparser.py



Testing

https://github.com/sympy/sympy/wiki/Running-tests
>>> import sympy
>>> sympy.test()
takes 2 hours on my MacBook Air

The relevant test is
>>> sympy.test("sympy/parsing/tests/test_latex.py")

1 comment:

  1. make docmac

    cd /scratch/sympy/sympy/parsing/latex
    java -jar /usr/local/lib/antlr-4.7.2-complete.jar LaTeX.g4 -no-visitor -no-listener -o _antlr
    python rename.py
    cd /scratch/sympy/
    python setup.py install
    exit

    cd sympy/
    git status
    git add sympy/parsing/latex/_antlr/latexlexer.py sympy/parsing/latex/_antlr/latexparser.py
    git commit -m "corrected version of Antlr from 4.8 to 4.7.2"
    git push

    ReplyDelete