LS4GAN Containers

A container image may be built locally with:

$ mkdir ls4gan
$ git clone https://github.com/LS4GAN/containers.git ls4gan/containers
$ cd ls4gan/containers/docker/<name>
$ docker build -t ls4gan/<name> .

Note, some images build on top of others that are also built from here. The naming convention illustrated above should be kept in mind when looking at a FROM line in a Dockerfile.

Each container has its own default command (CMD) of bash which is run as the argument to the entry point (ENTRYPOINT) of bash -c. Thus to get an interactive shell:

$ docker run -ti ls4gan/wirecell
#

Or to run a command provided by the image:

$ docker run -ti ls4gan/wirecell "wire-cell --help"
[...help message...]

To derive a Singularity container image from a Docker image:

$ singularity build ls4gan-<name>-latest.sif  $IMAGE_URL 

The $IMAGE_URL can be in one of several forms

local

docker-daemon://ls4gan/<name>:latest

docker hub

docker://ls4gan/<name>:latest

SDCC registry

It recommended to follow the illustrated naming convention so that some provenance is kept when sharing the resulting image file.

To run a Singularity container

$ singularity exec ls4gan-<name>-latest.sif "wire-cell --help"

It is recommended to name the Singularity image file following the convention so that when sharing these files their origin is hinted.

To run the default shell or a program in the container

$ singularity run  /srv/tmp/ls4gan-wirecell-latest.sif "wire-cell --help"
[ ...help message...]
$ singularity run  /srv/tmp/ls4gan-wirecell-latest.sif
Singularity> which wire-cell
/usr/local/bin/wire-cell

The ls4gan area on Docker Hub holds some of the images produced here. In the examples we will use the ls4gan/wirecell image. To get this image into your local docker, run:

$ docker pull ls4gan/wirecell:latest

Or, if you are building images and a docker login they may be uploaded with, eg:

$ docker push ls4gan/wirecell:latest

This image provides the Wire-Cell Toolkit C++ and Python and their externals. It is built on a minimal Debian with WCT and additional software installed under /usr/local/

This environment can be used to run any "stand-alone" wire-cell job or any of the wirecell-* Python CLIs. It also provides lots of Python goodies including Numpy, Matplotlib, ipython, JupyterLab (needing special docker run to see its ports).

It also provides snakemake so can be used to exercise the toyzero data generator. Using the derived Singularity image to enjoy easy access to native home directory files:

$ git clone https://github.com/LS4GAN/toyzero.git
$ cd toyzero
$ singularity run /srv/tmp/ls4gan-wirecell-latest.sif "snakemake -jall -p just_images"
$ tree data
[...generated data files...]

This container provides (will provide) support for running a toyzero pipeline as a single, ready-to-run job. It rides on top of the wirecell container.

This container provides (will provide) support for JupyterLab notebooks. It includes ls4gan-python, toytools and other Python

Outside the BNL network it is possible to access the registry by forwarding a local port via SSH to an internal HTTPS proxy. Basically, follow docker guidance on setting HTTP/HTTPS proxy. For example

# cat <<EOF > /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://127.0.0.1:3128"
Environment="HTTPS_PROXY=http://127.0.0.1:3128"
Environment="NO_PROXY=localhost,127.0.0.1,.home,haiku"
EOF
# systemctl daemon-reload
# systemctl restart docker

It is also helpful to add the internal IP address to /etc/hosts.

It should now be possible to login

❯ docker login registry.sdcc.bnl.gov

And the two steps to register and upload an image:

❯ docker tag ls4gan/wirecell registry.sdcc.bnl.gov/toyzero/wirecell
❯ docker push registry.sdcc.bnl.gov/toyzero/wirecell

Or, docker pull as above.

Currently building a Singularity image from a Docker image in Portus does not work. Expect an error like:

❯ singularity pull --docker-login docker://registry.sdcc.bnl.gov/ls4gan/toyzero/wirecell
Enter Docker Username: bvlbne
Enter Docker Password: 
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://registry.sdcc.bnl.gov/ls4gan/toyzero/wirecell: error pinging docker registry registry.sdcc.bnl.gov: Get "https://registry.sdcc.bnl.gov/v2/": dial tcp 130.199.148.226:443: i/o timeout

This may be due to offsite access restriction. In any case, it needs more checking.