pyiron.base.master.parallel module

class pyiron.base.master.parallel.GenericOutput[source]

Bases: collections.OrderedDict

Generic Output just a place holder to store the output of the last child directly in the ParallelMaster.

class pyiron.base.master.parallel.JobGenerator(job, no_job_checks=False)[source]

Bases: object

JobGenerator - this class implements the functions to generate the parameter list, modify the individual jobs according to the parameter list and generate the new job names according to the parameter list.

static modify_job(job, parameter)[source]
next()[source]

Iterate over the child jobs

Returns:new job object
Return type:GenericJob
parameter_list
parameter_list_cached
class pyiron.base.master.parallel.ParallelMaster(project, job_name)[source]

Bases: pyiron.base.master.generic.GenericMaster

MasterJob that handles the creation and analysis of several parallel jobs (including master and continuation jobs), Examples are Murnaghan or Phonon calculations

Parameters:
  • project (ProjectHDFio) – ProjectHDFio instance which points to the HDF5 file the job is stored in
  • job_name (str) – name of the job, which has to be unique within the project
.. attribute:: job_name

name of the job, which has to be unique within the project

.. attribute:: status
execution status of the job, can be one of the following [initialized, appended, created, submitted,
running, aborted, collect, suspended, refresh, busy, finished]
.. attribute:: job_id

unique id to identify the job in the pyiron database

.. attribute:: parent_id

job id of the predecessor job - the job which was executed before the current one in the current job series

.. attribute:: master_id

job id of the master job - a meta job which groups a series of jobs, which are executed either in parallel or in serial.

.. attribute:: child_ids

list of child job ids - only meta jobs have child jobs - jobs which list the meta job as their master

.. attribute:: project

Project instance the jobs is located in

.. attribute:: project_hdf5

ProjectHDFio instance which points to the HDF5 file the job is stored in

.. attribute:: job_info_str

short string to describe the job by it is job_name and job ID - mainly used for logging

.. attribute:: working_directory

working directory of the job is executed in - outside the HDF5 file

.. attribute:: path

path to the job as a combination of absolute file system path and path within the HDF5 file.

.. attribute:: version

Version of the hamiltonian, which is also the version of the executable unless a custom executable is used.

.. attribute:: executable

Executable used to run the job - usually the path to an external executable.

.. attribute:: library_activated

For job types which offer a Python library pyiron can use the python library instead of an external executable.

.. attribute:: server

Server object to handle the execution environment for the job.

.. attribute:: queue_id

the ID returned from the queuing system - it is most likely not the same as the job ID.

.. attribute:: logger

logger object to monitor the external execution and internal pyiron warnings.

.. attribute:: restart_file_list

list of files which are used to restart the calculation from these files.

.. attribute:: job_type
Job type object with all the available job types: [‘ExampleJob’, ‘SerialMaster’, ‘ParallelMaster’,
‘ScriptJob’, ‘ListMaster’]
.. attribute:: child_names

Dictionary matching the child ID to the child job name.

.. attribute:: ref_job

Reference job template from which all jobs within the ParallelMaster are generated.

.. attribute:: number_jobs_total

Total number of jobs

collect_logfiles()[source]

Collect the log files of the external executable and store the information in the HDF5 file. This method is currently not implemented for the ParallelMaster.

collect_output()[source]

Collect the output files of the external executable and store the information in the HDF5 file. This method has to be implemented in the individual meta jobs derived from the ParallelMaster.

copy()[source]

Copy the GenericJob object which links to the job and its HDF5 file

Returns:New GenericJob object pointing to the same job
Return type:GenericJob
copy_to(project=None, new_job_name=None, input_only=False, new_database_entry=True)[source]

Copy the content of the job including the HDF5 file to a new location

Parameters:
  • project (ProjectHDFio) – project to copy the job to
  • new_job_name (str) – to duplicate the job within the same porject it is necessary to modify the job name - optional
  • input_only (bool) – [True/False] to copy only the input - default False
  • new_database_entry (bool) – [True/False] to create a new database entry - default True
Returns:

GenericJob object pointing to the new location.

Return type:

GenericJob

create_child_job(job_name)[source]

Internal helper function to create the next child job from the reference job template - usually this is called as part of the create_jobs() function.

Parameters:job_name (str) – name of the next job
Returns:next job
Return type:GenericJob
from_hdf(hdf=None, group_name=None)[source]

Restore the ParallelMaster from an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object - optional
  • group_name (str) – HDF5 subgroup name - optional
interactive_ref_job_initialize()[source]

To execute the reference job in interactive mode it is necessary to initialize it.

is_finished()[source]

Check if the ParallelMaster job is finished - by checking the job status and the submission status.

Returns:[True/False]
Return type:bool
iter_jobs(convert_to_object=True)[source]

Iterate over the jobs within the ListMaster

Parameters:convert_to_object (bool) – load the full GenericJob object (default) or just the HDF5 / JobCore object
Returns:Yield of GenericJob or JobCore
Return type:yield
number_jobs_total

Get number of total jobs

Returns:number of total jobs
Return type:int
output_to_pandas(sort_by=None, h5_path='output')[source]

Convert output of all child jobs to a pandas Dataframe object.

Parameters:
  • sort_by (str) – sort the output using pandas.DataFrame.sort_values(by=sort_by)
  • h5_path (str) – select child output to include - default=’output’
Returns:

output as dataframe

Return type:

pandas.Dataframe

ref_job

Get the reference job template from which all jobs within the ParallelMaster are generated.

Returns:reference job
Return type:GenericJob
refresh_submission_status()[source]

Refresh the submission status - if a job ID job_id is set then the submission status is loaded from the database.

reset_job_id(job_id=None)[source]

Reset the job id sets the job_id to None as well as all connected modules like JobStatus and SubmissionStatus.

run_if_interactive()[source]

For jobs which executables are available as Python library, those can also be executed with a library call instead of calling an external executable. This is usually faster than a single core python job.

run_if_refresh()[source]

Internal helper function the run if refresh function is called when the job status is ‘refresh’. If the job was suspended previously, the job is going to be started again, to be continued.

run_static()[source]

The run_static function is executed within the GenericJob class and depending on the run_mode of the Parallelmaster and its child jobs a more specific run function is selected.

save()[source]

Save the object, by writing the content to the HDF5 file and storing an entry in the database.

Returns:Job ID stored in the database
Return type:(int)
set_input_to_read_only()[source]

This function enforces read-only mode for the input classes, but it has to be implement in the individual classes.

show_hdf()[source]

Display the output of the child jobs in a human readable print out

to_hdf(hdf=None, group_name=None)[source]

Store the ParallelMaster in an HDF5 file

Parameters:
  • hdf (ProjectHDFio) – HDF5 group object - optional
  • group_name (str) – HDF5 subgroup name - optional
write_input()[source]

Write the input files - this contains the GenericInput of the ParallelMaster as well as reseting the submission status.

pyiron.base.master.parallel.job_wrap_function(parameters)[source]