...
Internet is Not Available on Compute Nodes. Jupyter sessions will have to run on the compute nodes which do not have Internet access. This means that you will not be able to download files, clone a repo from GitHub, install packages, etc on the compute nodes. You will need to perform these operations on the login node node (e.g gc-prd-hpclogin1.rcs.griffith.edu.au) before starting the session. You can run commands which need Internet access on the login nodes (gc-prd-hpclogin1). Any files that you download while on the login node will be available on the compute nodes.
| No Format |
|---|
# from behind VPN if off-campus or on wireless
ssh snumber@gc-prd-hpclogin1.rcs.griffith.edu.au
module load anaconda3/2020.11
source activate snumber-tf-cpu #e.g source activate s123456-tf-cpu
jupyter-notebook --no-browser --port=8889 --ip=127.0.0.1
# note the last line of the output which will be something like
http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0
# leave the session running |
...
| No Format |
|---|
ssh -N -f -L localhost:8889:localhost:8889 snumber@n059.rcs.griffith.edu.au |
Running on a Compute Node via interative pbs
First, from the headlogin node (gc-prd-hpclogin1), request an interactive session on a compute node.
The command below requests 1 CPU-core with 4 GB of memory for 1 hour:
| No Format |
|---|
qsub -I -q workq |
...
-l select=1:ncpus=1:mem=4gb,walltime=1:00:00 |
Once
...
the
...
node
...
has
...
been
...
allocated,
...
run
...
the hostname command
...
to
...
get
...
the
...
name
...
of
...
the
...
node
...
(e.
...
g gc-prd-hpcn002)
.You should not be in the login node if you are inside the jobWithin the interactive session, you will be automatically transferred to an available compute node. If none available, you may have to wait till one becomes available.
On the compute node inside the interactive job, do the following:
| No Format |
|---|
module load anaconda3/2021.11 |
...
source activate myenv #e.g source activate s123456-tf-cpu |
...
jupyter-notebook --no-browser --port=8889 --ip=0.0.0.0 |
...
or jupyter-lab --no-browser --port=8889 --ip=0.0.0.0 #note the last line of the output which will be something like http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0 # leave the session running |
Next,
...
start
...
a
...
second
...
terminal
...
session
...
on
...
your
...
local
...
machine
...
(e.g.,
...
laptop)
...
and
...
setup
...
the
...
tunnel
...
as
...
follows:
In the command below, be sure to replace gc-prd-hpcn002 with the hostname of the node that pbs assigned to you.
| No Format |
|---|
ssh -N -f -L 8889:gc-prd-hpcn002:8889 |
...
snumber@gc-prd-hpclogin1.rcs.griffith.edu.au |
...
#e.g: ssh -N -f -L 8890:gc-prd-hpcn002:8890 s123456@gc-prd-hpclogin1.rcs.griffith.edu.au |
...
|
...
|
...
Note above,that be sure to replace gc-prd-hpcn002 withwe selected the hostnameLinux ofport the8889 node that pbs assignedto connect to you.
...
the notebook. If you don't specify the port, it will default to port 8888 but sometimes this port can be already in use either on the remote machine or the local one
(i.e., your laptop). If the port you selected is unavailable, you will get an error message, in which case you should just pick another one. It is best to keep it greater than 1024.
Consider starting with 8888 and increment by 1 if it fails, e.g., try 8888, 8889, 8890 and so on. If you are running on a different port then substitute your port number for 8889.
Lastly, open a web browser on your laptop/desktop and copy and paste the URL from the previous output:
| No Format |
|---|
http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0 |
...
Another example: |
...
http://127.0.0.1:8890/?token=8df8a9a79c00f0813055d48dfc79785c8ff6597cc0b1c456 |
Choose "New" then "Python 3" to launch a new notebook. Note that Jupyter may use a port that is different than the one you specified. This is why it is importimportant to copy and paste the URL.
When you are done, terminate the ssh tunnel on your local machine (desktop/laptop) by running lsof -i tcp:8889 to get the PID and then kill -9 <PID> (e.g., kill -9 6010).
...