Install pyiron on your cluster in 15 minutes!
Attention
This guide is assuming a linux environment on the compute cluster!
First, fetch mambaforge (mamba is essentially faster conda) install script from the web:
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
Note
If you already have a conda environment: conda install mamba
also works!
Execute the script to install mambaforge:
bash Mambaforge-Linux-x86_64.sh
Note
NOTE: There are systems with a very tight filequota/memory quota on the home directory. In that case, you may need to install on a different directory. Usually there is a /software or /group directory that users can have permanent storage for software on.
You can adjust where mamba is installed by changing the directory when it asks you where it should be installed.
In this example, we install mamba in a folder named /software/abc123/
.
Refresh/restart your shell:
source .bashrc
Now you have the option of installing pyiron in an environment with:
mamba create -n YOURENVNAME
Change YOURENVNAME
to your liking.
Then activate your environment with:
mamba activate YOURENVNAME
Call this to install pyiron:
mamba install -c conda-forge pyiron pyiron_contrib
This can take some time, so just hang tight.
7. Now, we create a pyiron_resources
folder. This can be placed anywhere, but here we place it in our home folder (e.g. /home/abc123
).
You can figure out the absolute path of your home directory is by calling echo $HOME
:
mkdir /home/abc123/pyiron_resources
Now, create our pyiron configuration file,
.pyiron
in the home folder. Paste the following lines into the file:
[DEFAULT]
RESOURCE_PATHS = /home/abc123/pyiron_resources, /software/abc123/mambaforge/envs/pyiron/share/pyiron
PROJECT_CHECK_ENABLED = False
#DISABLE_DATABASE = True
FILE = ~/pyiron.db
Note the RESOURCE_PATHS
contain two entries:
/home/abc123/pyiron_resources
/software/abc123/mambaforge/envs/pyiron/share/pyiron
RESOURCE_PATHS
tells pyiron where we are storing our executables, job scripts and queue configuration settings.
The first is the directory we just made. The second is where pyiron’s environment is located on the filesystem. You can find where it is using which python
with the environment activated, which yields something like:
/software/abc123/mambaforge/bin/python
And you can replace the bin/…
bit onwards with envs/YOURENVNAME/share/pyiron
Now enter the
pyiron_resources
folder and make thequeues
folder:
cd /home/abc123/pyiron_resources
mkdir queues
Configure the queue on your supercomputer (example for SLURM setup, for others/more advanced setups see pysqa docs). Edit/create a queue.yaml
file in the queues
folder, with contents of:
queue_type: SLURM
queue_primary: work
queues:
work: {cores_max: 128, cores_min: 1, run_time_max: 1440, script: work.sh}
express: {cores_max: 128, cores_min: 1, run_time_max: 1440, script: express.sh}
Change cores_max/cores_max/run_time_max
into something fitting your HPC queue.
In the above example, the jobs submitted using pyiron are limited to somewhere between 1-128 cores, and a run time of 1440 minutes (1 day).
You can usually find this information about how many resources are allowed usually on the information pages of your cluster. It usually looks something like this.
The queue_primary string (“work” in the above script) is the name of the queue. Replace all instances of work, if you would like to use something else as the queue_name.
To add more queues, simply add more entries like the express
entry and configure the queueing script template express.sh
accordingly.
Create the
work.sh
file in the samequeues
directory, modifyYOURACCOUNT
,YOURQUEUENAME
andYOURENVNAME
accordingly:
#!/bin/bash
#SBATCH --output=time.out
#SBATCH --job-name={{job_name}}
#SBATCH --chdir={{working_directory}}
#SBATCH --get-user-env=L
#SBATCH --account=YOURACCOUNT
#SBATCH --partition=YOURQUEUENAME
#SBATCH --exclusive
{%- if run_time_max %}
#SBATCH --time={{ [1, run_time_max]|max }}
{%- endif %}
{%- if memory_max %}
#SBATCH --mem={{memory_max}}G
{%- endif %}
#SBATCH --cpus-per-task={{cores}}
source /software/abc123/mambaforge/bin/activate YOURENVNAME
{{command}}
In general, for the most pain-free experience, just replace the {{…}} fields that are present in the above template with your existing working scripts.
i.e. Replace where you put the number of cores with {{cores}}`
and {{memory_max}}
, and so on, in your already working jobscripts to generate this template for pyiron.
Notice that the environment is activated in this example script using the source …/activate
line. Make sure you do this or the queueing system can’t see the environment in which you installed pyiron.
Congrats! We’re almost there.
Now to verify the installation is working; we will conduct a test LAMMPS calculation.
Install the conda-packaged version of LAMMPS:
mamba install -c conda-forge lammps
Create a python script
test.py
containing the following (anywhere, preferably wherever you usually do calculations, e.g./scratch
). Change the username in theos.system("squeue -u abc123")
to your user.
from pyiron_atomistics import Project
import os
pr = Project("test_lammps")
structure = pr.create.structure.bulk('Al', cubic=True).repeat(3)
job = pr.create.job.Lammps(job_name='my_job')
job.structure = structure
job.calc_md()
job.run()
# The line above issues a warning concerning the potential. You can find more info on the following page:
# https://pyiron.readthedocs.io/en/latest/source/notebooks/first_steps.html
print(job['output/generic/energy_tot'])
print("If a list of numbers is printed above, running calculations on the head node works!")
# Test the queue submission
job_new = job.copy_to(new_job_name="my_job_2")
job_new.run(run_mode="queue", delete_existing_job=True)
os.system("squeue -u abc123") # change abc123 to your username
print("If a queue table is printed out above, with the correct amount of resources, queue submission works!")
Call the script with
python test.py
If the script runs and the appropriate messages print out, you’re finished! Congratulations! You’re finished with the pyiron install.
If you’re experiencing problems, please click here for frequently encountered issues (coming soon) installation_errors
For more complex tasks, such as configuring VASP or utilising on-cluster module based executables please click here Installation.
Install and configure pyiron on a local machine so you can submit to remote HPCs
Attention
The recommended way to install pyiron is via the conda package manager in a Linux environment. So if you are using Windows we recommend installing the Windows subsystem for Linux <https://docs.microsoft.com/en-us/windows/wsl/install-win10> before you install pyiron and if you are on macOS X we recommend using a virtual machine/ virtual box. Native installations on both Windows and macOS X are possible, but functionality is limited. The following instructions assume a linux-like environment. Windows installs will have to go through the Anaconda setup.
If you have already installed pyiron on your cluster, and it works, we can proceed.
If not, start at the top of this page and finish that first.
To install pyiron on your local machine, first install
mamba
via:
cd /root
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
Execute the script to install mambaforge:
bash Mambaforge-Linux-x86_64.sh
Now install pyiron via:
bash Mambaforge-Linux-x86_64.sh
Now you have the option of installing pyiron in an environment with:
mamba create -n YOURENVNAME
Change YOURENVNAME
to your liking.
Then activate your environment with:
mamba activate YOURENVNAME
Call this to install pyiron:
mamba install -c conda-forge pyiron
Now, we create a
pyiron_resources
folder. This can be placed anywhere, but here we place it in our/root
folder.
mkdir /home/pyiron_resources
Create the pyiron configuration file,
.pyiron
in the home folder. Paste the following lines into the file:
[DEFAULT]
FILE = ~/pyiron.db
RESOURCE_PATHS = /home/pyiron_resources
Now enter the
pyiron_resources
folder and make thequeues
folder:
cd /home/pyiron_resources
mkdir queues
Copy the contents of the queues folder from your remote cluster into the folder.
So now, there should be a queue.yaml
file and a work.sh
file in there.
Now we configure a
ssh_key
for the connection between your cluster/HPC and your local machine.
Call ssh-keygen
:
root@HanLaptop:~# ssh-keygen
Generating public/private rsa key pair.
When it prompts you with Enter file in which to save the key (/root/.ssh/id_rsa):
, input:
/root/.ssh/id_rsa_YOURHPC
Rename the id_rsa_YOURHPC
accordingly.
When it prompts you for the passphrases, just press Enter
twice - we don’t need a passphrase:
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
And now, the final output in your local terminal looks something like:
root@HanLaptop:~# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/id_rsa_YOURHPC
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa_YOURHPC
Your public key has been saved in /root/.ssh/id_rsa_YOURHPC.pub
The key fingerprint is:
SHA256:AVNJ4qG55/fevDfgUb3OUWDePelBBiSJBtCEiicSCjI root@laptop
The key's randomart image is:
+---[RSA 3072]----+
| .X=+...oo. |
|E = *.o .. oo |
|+o. + . o oo+o|
|oo o . . o+=|
|. o . . S .. =|
| o o * |
| . . . oo .|
| . . o. oo |
| .o +o . |
+----[SHA256]-----+
Now, copy the contents of
id_rsa_YOURHPC.pub
over to the remote cluster into the$HOME/.ssh/authorized_keys
.
If the file is not empty, make sure that there is an empty line in between entries.
Check that the key works by checking that we can ssh
into the remote cluster on your local terminal without a password:
ssh abc123@gadi.nci.org.au
If it works, it means that the ssh key works, and we can proceed.
Edit the
queue.yaml
file:
queue_type: REMOTE
queue_primary: work
ssh_host: gadi.nci.org.au
ssh_username: abc123
known_hosts: /root/.ssh/known_hosts
ssh_key: /root/.ssh/id_rsa_YOURHPC
ssh_remote_config_dir: /home/abc123/pyiron_resources/queues/
ssh_remote_path: /scratch/a01/abc123/pyiron/
ssh_local_path: /root/pyiron_remote_data/
ssh_continous_connection: True
queues:
work: {cores_max: 128, cores_min: 1, run_time_max: 1440, script: work.sh}
express: {cores_max: 128, cores_min: 1, run_time_max: 1440, script: express.sh}
Replace the following fields accordingly:
queue_primary
: The primary queue that you use. Must be present at the bottom queues
field.
ssh_host
: The host address of your remote cluster.
E.g. If you sign in usually with ssh abc123@gadi.nci.org.au
, it is gadi.nci.org.au
.
ssh_username
: The username that you usually sign in with.
E.g. If you sign in usually with ssh abc123@gadi.nci.org.au
, it is abc123
.
known_hosts
: The directory where you store your known_hosts
locally. If you don’t know what this is, you most likely don’t need to change this field.
ssh_key
: The ssh_key
that you generated in the previous step.
ssh_remote_config_dir
: Path to where you have your queues configured on the remote cluster.
ssh_remote_path
: Path to where you want to run the calculations on the remote cluster.
ssh_local_path
: Local path to place the calculations you’ve fetched the results from the cluster on your local machine.
ssh_continous_connection
: Whether or not to use a single SSH connection or multiple ones (use this if your connection is unreliable).
The entries underneath queues
should read the same as what you have in the queue.yaml file
in the remote cluster as you have previously configured:
Now, at this point, the submission should work. Let’s test a submission of a small job. On the local machine create a python script:
Attention
pyiron
must be present in the environment that is present after you initialise a shell in the remote machine! If it is not, pyiron will fail to initialise the calculation!
To make pyiron the default environment after you initialise the shell, add the following line to your .bashrc
:
source /software/abc123/mambaforge/bin/activate pyiron
Adjust the above path to the appropriate path such that it can activate a python environment containing pyiron
.
from pyiron_atomistics import Project
import os
pr = Project("test_lammps")
job = pr.create.job.Lammps(job_name='my_job_remote')
structure = pr.create.structure.bulk('Al', cubic=True).repeat(3)
job.structure = structure
job.calc_md()
job.server.queue = "work" # Your queue server name
job.server.cores = 2
job.server.memory_limit = 2
# The line above issues a warning concerning the potential. You can find more info on the following page:
# https://pyiron.readthedocs.io/en/latest/source/notebooks/first_steps.html
job.run()
Once the job is done on the queue, we can fetch the job back using:
pr = Project("test_lammps")
job_name = "my_job_remote"
pr.wait_for_job(pr.load(job_specifier=job_name))
And then verify that the fetched job has results associated with it:
job = pr.load(job_name)
print(job["output/generic/energy_tot"])
If some list of numbers prints out in the output, then the calculation was successful!
For more complex setups - such as those involving multiple remote clusters and one host machine, please see Installation.