You can get access to the Linux server cafmaster2.econ.au.dk, which is a master node for 3 linux nodes in a cluster. You should only use cafmaster2 for small test jobs, and submitting large jobs to the cluster using the command sbatch, see below. The 3 nodes are
The cluster is used for heavy scientific computation and access to the TAQ and CRSP databases.
It can be used for parallel computing using OpenMPI.
You must use your UNI username/password.
Faculty: You must send an email to be added as a user to the cluster, email mbj@econ.au.dk
Students: Your supervisor must send an email having you added to the cluster.
You login to cafmaster2.econ.au.dk in order to edit files, compiling jobs using Intel Fortran or gcc, running small jobs interactively, accessing the databases.
You submit large batch jobs to all servers from cafmaster2 using slurm queue system - see below.
You can access cafmaster2 using ssh, or rdp (remote desktop protocol). If you are not connected using a university cabled
network you must first create a VPN tunnel to the network using: remote.au.dk.
From Windows Computers we are using the following software to access the servers
You will not have a graphical interface to the Linux server if you use putty.
Your can transfer files using:
You may experience problems with firewalls, either your own or AU firewalls. If you are not able to connect using these protocols, please send an email with information about your network and the error message.
You get network information as follows: On Windows, use cmd to get a dos window, use the command ipconfig /all.
You must be on the AU NET if you are using remote desk top and mounting /home.
You can get access to the Aarhus University net using the vpn connection https://remote.au.dk. Search the web for help, au.dk: vpn access.
The vpn connection will interrupt any local network connection. Hence if you want to stay connected for longer periods you may want to use a spare computer or dual installed operating system.
Winscp is a free file explorer which you can download from https://winscp.net/eng/download.php. You get your local files in one window and the remote (Linux) files in the other window. You can move and copy files as in Windows and by right-clicking on a file you can open a remote file for editing. Moved and copied text files are automatically converted between windows and Linux format.
You run interactive jobs by writing the program name in an xterm window. If you want to run the job in the background you either create a screen or add & to the command. You can create an xterm window by right-clicking on the desktop.
You run batch jobs by
We are using the Centos version of Linux. You run installed programs from a terminal using the commands shown below: For some programs you can open a graphic user interface, gui.
For command line jobs: matlab -nodesktop - nosplash �r <your-Matlab-program>
For GUI/interactive jobs: matlab
For slurm:
1. Change to the directory containing <your-Matlab-program>. This directory must be in your matlab path. Start Matlab interactively, choose file, choose set path and include the directory.
2. create a s job_script, <myjob>.sh, using an editor with the line: matlab -nodesktop - nosplash -r <your-Matlab-program>
3. sbatch <myjob>.sh
matlab -nodesktop -help shows options
For command line jobs: oxl [arguments for Ox program] <your-Ox-program>
For GUI/interactive jobs: Oxmetrics
Parallel package is currently not installed.
For slurm:
1. create a s script file, <myjob>.sh, using an editor with the line: oxl [arguments for Ox program] <your-Ox-program>
2. sbatch <myjob>.sh
oxl -help shows options
For command line: R CMD BATCH� myscript.R [outfile]
For slurm:
1. create a s script file, <myjob>.sh, using an editor with the line: R CMD BATCH <your-R-program>
2. sbatch <myjob>.sh
1. If you are running large jobs you should always login to cafmaster2.econ.au.dk and use the queue system.
2. The SmallJob partition is submitting jobs to cafnode1.
3. The BigJob partition is submitting jobs to creates1, creates2 and creates3.
4. Your job is assigned a job_id, which is used to identify output and errors, in squeue, and scancel job_id which deletes your job from the processors.
5. You can follow your job using the command squeue �u user , and monitor the utilization of the nodes using the commands sinfo.
6. Slurm is described at https://slurm.schedmd.com/documentation.html and the commands are illustrated at https://support.ceci-hpc.be/doc/_contents/QuickStart/SubmittingJobs/SlurmTutorial.html.
7. SLURM_SUBMIT_DIR is the directory where the sbatch command was submitted.
This is an example of a batch file, which submits the file sleep to the queue system. The batch file is named slurm_job.sh
#!/bin/bash
#SBATCH --job-name=Testjob
#SBATCH --partition=BigJob
#SBATCH -w creates1 #spec node
#SBATCH --mem=1G
#SBATCH --ntasks=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --time=240:00:00
#SBATCH -o slurm.%N.%j.out # STDOUT
#SBATCH -e slurm.%N.%j.err # STDERR
cd $SLURM_SUBMIT_DIR
./sleep
assuming that sleep is located at the same directory as slurm_job.sh.