Griffith HPC
Griffith HPC Computing Resource
The High Performance Computing environment consists of high-end systems used for executing complex number crunching applications for research. Griffith users have access to the following systems:
Griffith HPC cluster
Awoonga cluster
QRIScloud Special Offering
Gowonda cluster
Microsoft Azure cloud
National Computational Infrastructure (NCI)
Pawsey Supercomputing Centre
Griffith HPC cluster
Griffith has funded the upgrade of the HPC cluster and we are excited to announce it has finally come into service. There are 6 nodes in total with a total of 432 cores for compute. Each node has a total of 192G of memory and 2 X Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz with 72 cores. It runs RedHat 7.6 and is connected to a fast storage through Network File Share (NFS) and infiniband interconnect.
If you wish to use the new cluster, please follow the link and request access on the new cluster. We will add you to the LML – GHPC list to enable you to log into the new cluster. Further information on how to connect to the new cluster and usage (batching system, available applications, etc) would be sent at that time.
If you need any help or have questions you can contact us by creating a support request.
We sincerely hope you will benefit from the much-improved computing resources now available to all Griffith users. We welcome any feedback specially related to any improvements or additional resources required by your research.
Awoonga Cluster
Griffith has partnered with QCIF which operates the Awoonga HTC cluster (1000 cores). Griffith users can request an account on Awoonga cluster using this link to create an account on the awoonga cluster. Please click the link "Register to use Awoonga”. The number of cores available to a single job is limited to 24 on one node. It is not possible to create an MPI job using multiple nodes. Hence all mpi jobs should be limited to 24 cores on a single node. It is a Symmetric Multi Processing (SMP) system having a maximum of 24 cores per node and sharing the same memory and is managed by one operating system. The home directory and software directory is shared across all nodes.
Ref: https://www.qriscloud.org.au/support/qriscloud-documentation/92-awoonga-user-guide
In general, you have a question or problem regarding Awoonga, please raise a new QRIScloud support request by emailing support@qriscloud.org.au. For further support options, please visit https://www.qriscloud.org.au/support.
QRIScloud Special Offering
QRIScloud offers special nodes to Griffith users.
To request a special node, please follow this link and select "Use specialised compute"
In general, you have a question or problem regarding special nodes on QRIScloud, please raise a new QRIScloud support request by emailing support@qriscloud.org.au. For further support options, please visithttps://www.qriscloud.org.au/support.
Flashlite node
This is a large memory cluster named Flashlite (large memory and fast disks in a non-virtualized environment). FlashLite is a research computer that has been designed and optimised for data intensive computing. FlashLite will support applications that need large amounts of primary memory along with very high performance secondary memory. Each of the 68 nodes of FlashLite is equipped with 512 Gbytes of main memory and 4.8TB of solid-state drive (SSD). The operating software supports various programming paradigms including message passing (MPI), shared memory (OpenMP), and virtual symmetric multiprocessor (vSMP). The vSMP mode aggregates nodes to produce virtual machines with very large "main" memory address space.
Large memory node
QRIScloud operates a small number of compute nodes that can accommodate instances with up to 1TB memory. These can be made available in blocks of time of two weeks as instances with up to 60 vCPUs and 900Gb of memory. Instances can be provisioned with large storage volumes to hold working data.
GPU node
QRIScloud operates a small number of compute nodes that contain a Tesla K20m GPU. These can be made available in blocks of time of two weeks as instances. Instances can be provisioned with storage volumes to hold working data.
Elastic compute
Normal Nectar resources are often in short supply, leading to problems with launching instances. QRIScloud has set aside some limited capacity to allow users to create instances with large numbers of vCPUs with a relatively short (up to 7 days) lifetime (elastic compute).
QCIF's share of NCI
QCIF has a share in time on Raijin and is accepting applications all year round. To request QCIF's share of NCI, follow this link and select "Apply for QCIF's NCI share "
Commercial Cloud Resources
Commercial cloud resources are suitable for researchers needing large resources (e.g memory requests above 150GB, latest GPU cards, etc). It has the advantage of being highly scalable. It is also suitable where dedicated nodes are required without having to share resources with other users. It is also suitable if a large windows environment is required with memory over 64GB and GPU support.
Cloud resources like Azure and AWS compute are available to Griffith researchers on cost recovery basis. Please contact us on eresearch-support@griffith.edu.au for further details and help with setting this up.
For Azure cost calculator, please refer to: https://azure.microsoft.com/en-au/pricing/calculator/. The AWS costing calculator can be found here. To give an idea of the cost, in the pilot run in 2018-19, Azure Data Science Image with ArcGIS Pro VM (Windows-based GPU using traditional batch HPC architecture, 2 x VMs: 8 vCPU, 56GB memory and 2 x VMs: 12 vCPU, 112GB memory) cost $5,652 for 4,196 hours consumed.
There are other cloud resources like google's cloud Tensor Processing Units (TPUs). Cloud TPU resources accelerate the performance of linear algebra computation. The advantages of TPUs include the minimising of time-to-accuracy when you train large, complex neural network models. Models that previously took weeks to train on other hardware platforms can converge in hours on TPUs.
National Computational Infrastructure (NCI)
The National Computational Merit Allocation Scheme (NCMAS) provides researchers with access to Australia’s major national computational facilities, including Raijin. The main call for applications is made annually in October for allocations to start the following January for up to 12 months.
Australia’s national research computing service, the National Computational Infrastructure (NCI), provides world-class, high-end services to Australia’s researchers, the primary objectives of which are to raise the ambition, impact, and outcomes of Australian research through access to advanced, computational and data-intensive methods, support, and high-performance infrastructure.
NCI's peak system,Raijin, is a Fujitsu Primergy high-performance, distributed-memory cluster which entered production use in June 2013. It comprises more than 50,000 cores (Intel Xeon Sandy Bridge technology, 2.6 GHz), 160 TBytes of main memory, Infiniband FDR interconnect and 10 PBytes of usable fast filesystem (for short-term scratch space).The unit of shared memory parallelism is the node, which comprises dual 8-core Intel Xeon (Sandy Bridge 2.6 GHz) processors, i.e., 16 cores. The memory specification across the nodes is heterogeneous in order to provide a configuration capable of accommodating the requirements of most applications, and providing also for large-memory jobs. Raijin is particular suited to large scale MPI jobs which use less than 2GB per core and require low latency interconnects.
Ref: https://opus.nci.org.au/display/Help/Getting+Started+at+NCI
Pawsey Supercomputing Centre
The Pawsey Supercomputing Centre operates multiple supercomputers, data-intensive machines and storage systems that use the most advanced technologies available. The Pawsey Supercomputing Centre supports researchers in Western Australia and across the nation with an array of capabilities encompassing supercomputing, data and visualisation services. To gain access to the resources of the Pawsey Supercomputing Centre, to find out about training or to receive help with an issue, please visit the Pawsey User Portal page
Ref: https://www.pawsey.org.au/