Frequently Asked Questions¶

About ARIS¶

Acknowledge usage of ARIS¶

When the work carried out using GRNET’s HPC resources leads to publications, acknowledge of GRNET support is requested, and a copy of each should be sent to GRNET. In addition to those possible publications, the user will also have to provide any material with dissemination purposes as requested by GRNET. Users must quote and acknowledge the use of GRNET resources in all publications related to their production and development projects as follows:

“This work was supported by computational time granted from the National Infrastructures for Research and Technology S.A. (GRNET S.A.) in the National HPC facility - ARIS - under project ID ###”

The ID for your project is derived from your account and project name as follows:

<Account_Name>-<Project_Name>

You can check your project information with the command mybudget

$ mybudget 

=======================================================================
Core Hours Allocation Information for account : <Account_Name>
=======================================================================

What is a core-hour ?¶

Usage of ARIS resources is accounted in core-hours. One core-hour equals one CPU core being used for the duration of one hour of execution time. Time is measured as the elapsed wall clock time for the job start to the job finish and NOT as the actual CPU time.

Core Hours Calculator Example

320 cores x 24 Hours = 7680 Core Hours

16 Nodes x 20 Cores x 24 Hours = 7680 Core Hours

NOTE : If your job use compute nodes exclusively, usage is always equal to the total number of CPU cores on the allocated nodes times the execution time, regardless of the actual number of node slots allocated to the job.

How many cores and memory are available on each node ?¶

$ sinfo -o "%15N %10c %10m"

NODELIST        CPUS       MEMORY    
node[001-426]   20         57344

For more information about hardware components please check the Hardware Overview

About my account¶

How many core hours have i consumed ?¶

Use command myreport to check your account, core hours and energy utilization.

How many core hours are available for my account ?¶

Command mybudget informs your allocated and remaining core hours.

SLURM tips¶

Where can i find a SLURM submission script example ?¶

You can use the job script template http://doc.aris.grnet.gr/scripttemplate/

Which nodes are allocated to my job?¶

The environment variable $SLURM_NODELIST contains the names of the nodes. The format is something like: node[005-007].

Using the command scontrol you can obtain the node names, one name per line:

$ scontrol show hostname $SLURM_NODELIST
node005
node006
node007

How can I cancel a job ?¶

Use the scancel <jobid> command

More help: Cancel jobs

My job is on pending state why ?¶

Please check the common reasons for awaiting jobs. Pending Jobs

My job submission failed why ?¶

sbatch: error: Batch job submission failed: Job violates accounting/QOS policy
(job submit limit, user's size and/or time limits)

A common reason for a submission fail is due to using invalid account. Check that you have defined your account in your SLURM script.

#SBATCH --account pr0000 # Accounting project

Ther is no default account if you have multiple allocations for your username.

How can I set the number of OPENMP threads ?¶

You need to set the environment variable $OMP_NUM_THREADS in your submission script.

...
#SBATCH --cpus-per-task=1     # Threads per task
...
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

How can I determine the memory usage of my application?¶

For running jobs, use sstat <jobid>, for finished jobs use sacct

Example:

sacct --format JobID,jobname,NTasks,nodelist,MaxRSS,MaxVMSize,AveRSS,AveVMSize

RSS: resident set size (RSS) is the portion of memory occupied by a process that is held in main memory RAM.
VMS: virtual memory size (VMS) It includes all memory that the process can access, including memory that is swapped out and memory that is from shared libraries.

More help:

man sstat
man sacct

How can I convert a submission script from another job scheduler into Slurm ?¶

To convert

PBS/Torque
LSF
SGE
LoadLeveler

into SLURM please use the Rosetta Stone of Workload Managers