6.2 Uploading and security

Building the website from scratch for local checking

Initial setup:

Create directories:

$HOME/lilypond/
$HOME/lilypond/media/
$HOME/lilypond/trusted-scripts/

To reduce the CPU burden on the shared host (as well as some security concerns), the ’Documentation/pictures/’ and ’Documentation/web/ly-examples/’ directories are **not** compiled. We will do this ourselves right now.

Go to your lilypond build directory. make doc.

Set up some variables (you’ll only do this once:)

BUILD_DIR=$HOME/lilypond-git
PICS=$BUILD_DIR/Documentation/pictures/
EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/

Copy files over:

cp -r $PICS $HOME/lilypond/media/pictures
cp -r $EXAMPLES $HOME/lilypond/media/ly-examples

Get the scripts you need. First define these variables:

GIT=$HOME/lilypond-git
DEST=$HOME/lilypond/trusted-scripts

Then do the copying:

cp $GIT/make/website.make $DEST/website.make
cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
cp $GIT/scripts/build/extract_texi_filenames.py  $DEST/extract_texi_filenames.py
cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
cp $GIT/scripts/build/website_post.py $DEST/website_post.py
cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess

Delete your build directory (or maybe just rename your build directory to build-old).

cd $HOME/lilypond

Run

make -f ../lilypond-git/make/website.make WEBSITE_ONLY_BUILD=1 \
  TOP_SRC_DIR=$HOME/lilypond-git/ \
  PYTHONPATH=$HOME/lilypond-git/python \
  TEXI2HTML_PROGRAM=texi2html \
  website

The website should be at:

$HOME/lilypond/out-website/website/index.html

Building the online website

The website is generated hourly by user graham the host lilypond.org. For security reasons, we do not use the makefiles and scripts directly from git; copies of the relevant scripts are examined and copied to ~graham/lilypond/trusted-scripts/

Initial setup

You should symlink your own ‘~/lilypond/’ to ‘~graham/lilypond/’

If this directory does not exist, make it. Git master should go in ‘~/lilypond/lilypond-git/’ but make sure you enable:

git config core.filemode false

If you have created any files in ‘~graham/lilypond/’ then please run:

chgrp lilypond ~graham/lilypond/ -R
chmod 775 ~graham/lilypond/ -R

To reduce the CPU burden on the shared host (as well as some security concerns), the ‘Documentation/pictures/’ and ‘Documentation/web/ly-examples/’ directories are not compiled. You need to upload them, and if they ever change, a user in the lilypond group must upload them to ‘~graham/lilypond/media’ on the host.

Upload latest pictures/ and ly-examples/ (local script):

### upload-lily-web-media.sh
#!/bin/sh
BUILD_DIR=$HOME/src/build-lilypond

PICS=$BUILD_DIR/Documentation/pictures/out-www/
EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/

cd $BUILD_DIR
rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples

Normal maintenance

Get latest source code:

### update-git.sh
#!/bin/sh
cd $HOME/lilypond/lilypond-git
git fetch origin
git merge origin/master

Check for any updates to trusted scripts / files:

### check-git.sh
#!/bin/sh
GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
diff -u $DEST/website.make $GIT/make/website.make
diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess

If the changes look ok, make them trusted:

### copy-from-git.sh
#!/bin/sh
GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
cp $GIT/make/website.make $DEST/website.make
cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
cp $GIT/scripts/build/website_post.py $DEST/website_post.py
cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess

Build the website:

### make-website.sh
#!/bin/sh
DEST=$HOME/web/
BUILD=$HOME/lilypond/build-website
mkdir -p $BUILD
cd $BUILD
cp $HOME/lilypond/trusted-scripts/website.make .

make -f website.make WEBSITE_ONLY_BUILD=1 website
rsync -raO $BUILD/out-website/website/ $DEST/website/
cp $BUILD/out-website/pictures $DEST
cp $BUILD/out-website/.htaccess $DEST

Cronjob to automate the trusted portions:

# website-rebuild.cron
11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1

Additional information

Some information about the website is stored in ‘~graham/lilypond/*.txt’; this information should not be shared with people without trusted access to the server.

<< Back to Documentation Index

LilyPond — Contributor’s Guide