WorkBalance

Short Description

Computes several metrics for workload balance per processor

Description

The idea here is to compute per-processor metrics to help in determining the quality of a partitioning.

Currently computes: number of local elements, nodes, dofs and partition sides. The partition sides are the sides of elements that are on processor boundaries (also known as the "edge-cuts" in partitioner lingo). Also computes the "surface area" of each partition (physically, how much processor boundary each partitioning has).

commentnote:Vector names

The names of the vectors declared by WorkBalance are the names of the "balances" requested.

HardwareID

WorkBalance will now also compute the number of sides and the surface area for the partition on each compute node (called "hardware_id" here) in the cluster. This gives the amount of "inter-node" communication. Use of a hierarchical partitioner (like the one available in PetscExternalPartitioner) can help reduce inter-node communication.

For instance, here is a 1600x1600 mesh partitioned to run on 64 nodes, each having 36 processors (2304 processors total). Using WorkBalance and VectorPostprocessorVisualizationAux we can visually see how much inter-node communication there is and quantify it.

Visualization of inter-node communication. Left: Parmetis, Right: Hierarchical. Parmetis hardware_id_surface_area: 66. Hierarchical hardware_id_surface_area: 39.

Input Parameters

  • balancesnum_elems num_nodes num_dofs num_partition_sides partition_surface_area num_partition_hardware_id_sides partition_hardware_id_surface_areaWhich metrics do you want to use to represent word balance

    Default:num_elems num_nodes num_dofs num_partition_sides partition_surface_area num_partition_hardware_id_sides partition_hardware_id_surface_area

    C++ Type:MultiMooseEnum

    Options:num_elems, num_nodes, num_dofs, num_partition_sides, partition_surface_area, num_partition_hardware_id_sides, partition_hardware_id_surface_area

    Controllable:No

    Description:Which metrics do you want to use to represent word balance

  • contains_complete_historyFalseSet this flag to indicate that the values in all vectors declared by this VPP represent a time history (e.g. with each invocation, new values are added and old values are never removed). This changes the output so that only a single file is output and updated with each invocation

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Set this flag to indicate that the values in all vectors declared by this VPP represent a time history (e.g. with each invocation, new values are added and old values are never removed). This changes the output so that only a single file is output and updated with each invocation

  • execute_onTIMESTEP_ENDThe list of flag(s) indicating when this object should be executed, the available options include FORWARD, ADJOINT, HOMOGENEOUS_FORWARD, ADJOINT_TIMESTEP_BEGIN, ADJOINT_TIMESTEP_END, NONE, INITIAL, LINEAR, NONLINEAR, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM.

    Default:TIMESTEP_END

    C++ Type:ExecFlagEnum

    Options:FORWARD, ADJOINT, HOMOGENEOUS_FORWARD, ADJOINT_TIMESTEP_BEGIN, ADJOINT_TIMESTEP_END, NONE, INITIAL, LINEAR, NONLINEAR, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM

    Controllable:No

    Description:The list of flag(s) indicating when this object should be executed, the available options include FORWARD, ADJOINT, HOMOGENEOUS_FORWARD, ADJOINT_TIMESTEP_BEGIN, ADJOINT_TIMESTEP_END, NONE, INITIAL, LINEAR, NONLINEAR, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM.

  • parallel_typeREPLICATEDSet how the data is represented within the VectorPostprocessor (VPP); 'distributed' indicates that data within the VPP is distributed and no auto communication is performed, this setting will result in parallel output within the CSV output; 'replicated' indicates that the data within the VPP is correct on processor 0, the data will automatically be broadcast to all processors unless the '_auto_broadcast' param is set to false within the validParams function.

    Default:REPLICATED

    C++ Type:MooseEnum

    Options:DISTRIBUTED, REPLICATED

    Controllable:No

    Description:Set how the data is represented within the VectorPostprocessor (VPP); 'distributed' indicates that data within the VPP is distributed and no auto communication is performed, this setting will result in parallel output within the CSV output; 'replicated' indicates that the data within the VPP is correct on processor 0, the data will automatically be broadcast to all processors unless the '_auto_broadcast' param is set to false within the validParams function.

  • prop_getter_suffixAn optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

    C++ Type:MaterialPropertyName

    Controllable:No

    Description:An optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.

  • sync_to_all_procsFalseWhether or not to sync the vectors to all processors. By default we only sync them to processor 0 so they can be written out. Setting this to true will use more communication, but is necessary if you expect these vectors to be available on all processors

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Whether or not to sync the vectors to all processors. By default we only sync them to processor 0 so they can be written out. Setting this to true will use more communication, but is necessary if you expect these vectors to be available on all processors

  • systemALLThe system(s) to retrieve the number of DOFs from (NL, AUX, ALL). Default == ALL

    Default:ALL

    C++ Type:MooseEnum

    Options:ALL, NL, AUX

    Controllable:No

    Description:The system(s) to retrieve the number of DOFs from (NL, AUX, ALL). Default == ALL

  • use_interpolated_stateFalseFor the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

    Default:False

    C++ Type:bool

    Controllable:No

    Description:For the old and older state use projected material properties interpolated at the quadrature points. To set up projection use the ProjectedStatefulMaterialStorageAction.

Optional Parameters

  • allow_duplicate_execution_on_initialFalseIn the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

    Default:False

    C++ Type:bool

    Controllable:No

    Description:In the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).

  • control_tagsAdds user-defined labels for accessing object parameters via control logic.

    C++ Type:std::vector<std::string>

    Controllable:No

    Description:Adds user-defined labels for accessing object parameters via control logic.

  • enableTrueSet the enabled status of the MooseObject.

    Default:True

    C++ Type:bool

    Controllable:Yes

    Description:Set the enabled status of the MooseObject.

  • execution_order_group0Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

    Default:0

    C++ Type:int

    Controllable:No

    Description:Execution order groups are executed in increasing order (e.g., the lowest number is executed first). Note that negative group numbers may be used to execute groups before the default (0) group. Please refer to the user object documentation for ordering of user object execution within a group.

  • force_postauxFalseForces the UserObject to be executed in POSTAUX

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Forces the UserObject to be executed in POSTAUX

  • force_preauxFalseForces the UserObject to be executed in PREAUX

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Forces the UserObject to be executed in PREAUX

  • force_preicFalseForces the UserObject to be executed in PREIC during initial setup

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Forces the UserObject to be executed in PREIC during initial setup

  • outputsVector of output names where you would like to restrict the output of variables(s) associated with this object

    C++ Type:std::vector<OutputName>

    Controllable:No

    Description:Vector of output names where you would like to restrict the output of variables(s) associated with this object

  • use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

    Default:False

    C++ Type:bool

    Controllable:No

    Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

Advanced Parameters

Input Files