| Table of Contents |
|---|
...
To create containers, we use public.docker.itc.griffith.edu.au
Simple "singularity pull" with s3proxy stopped working for docker hubs around Oct 2020. This is due to the changes (limits) docker introduced. To avoid hitting the limits, Griffith team implemented steps to stop direct access through the s3proxy http proxy to the docker hub registry (see https://www.docker.com/blog/understanding-inner-loop-development-and-pull-rates/ for notes)
...
| No Format |
|---|
1. Find the Docker image ID.
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest bf756fb1ae65 5 months ago 13.3kB
godlovedc/lolcow latest 577c1fe8e6d8 2 years ago 241MB
2. Create a tarball of the Docker image.
For the Docker image you want to port to Griffith HPC,
for example, godlovedc/lolcow with an image ID of 577c1fe8e6d8,
create a tarball using the docker save command:
docker save 577c1fe8e6d8 -o lolcow.tar
3. Copy the tarball to Griffith HPC
Use scp or winscp or cyberduck etc
scp lolcow.tar 10.250.250.3:/tmp
4. Convert the tarball to a Singularity image.
module load singularity
singularity build --sandbox lolcow docker-archive://lolcow.tar
If the tarball is not in the current working directory, specify the path, for example, /tmp:
singularity build --sandbox lolcow docker-archive:///tmp/lolcow.tar
singularity build --sandbox pytorch docker-archive:////sw/Containers/docker/pytorchnew.tar
In this example, lolcow is the directory name of the Singularity image.
5. Run the Singularity sandbox as usual for testing. Once testing is over, use it in a pbs script (see example above).
For example:
singularity shell lolcow
singularity exec lolcow cowsay hello
singularity run lolcow |
...