TIMPI::Communicator Class Reference

Encapsulates the MPI_Comm object. More...

#include <communicator.h>

Inheritance diagram for TIMPI::Communicator:

Public Types
enum	SendMode { DEFAULT =0, SYNCHRONOUS }
	Whether to use default or synchronous sends? More...
enum	SyncType { NBX, ALLTOALL_COUNTS, SENDRECEIVE }
	What algorithm to use for parallel synchronization? More...

Public Member Functions
	Communicator ()
	Default Constructor. More...
	Communicator (const communicator &comm)
	Communicator (const Communicator &)=delete
Communicator &	operator= (const Communicator &)=delete
	Communicator (Communicator &&)=default
Communicator &	operator= (Communicator &&)=default
	~Communicator ()
void	split (int color, int key, Communicator &target) const
void	split_by_type (int split_type, int key, info i, Communicator &target) const
void	duplicate (const Communicator &comm)
void	duplicate (const communicator &comm)
communicator &	get ()
const communicator &	get () const
MessageTag	get_unique_tag (int tagvalue=MessageTag::invalid_tag) const
	Get a tag that is unique to this Communicator. More...
void	reference_unique_tag (int tagvalue) const
	Reference an already-acquired tag, so that we know it will be dereferenced multiple times before we can re-release it. More...
void	dereference_unique_tag (int tagvalue) const
	Dereference an already-acquired tag, and see if we can re-release it. More...
void	clear ()
	Free and reset this communicator. More...
Communicator &	operator= (const communicator &comm)
processor_id_type	rank () const
processor_id_type	size () const
void	send_mode (const SendMode sm)
	Explicitly sets the SendMode type used for send operations. More...
SendMode	send_mode () const
	Gets the user-requested SendMode. More...
void	sync_type (const SyncType st)
	Explicitly sets the SyncType used for sync operations. More...
void	sync_type (const std::string &st)
	Sets the sync type used for sync operations via a string. More...
SyncType	sync_type () const
	Gets the user-requested SyncType. More...
void	barrier () const
	Pause execution until all processors reach a certain point. More...
void	nonblocking_barrier (Request &req) const
	Start a barrier that doesn't block. More...
template<typename T >
bool	verify (const T &r) const
	Verify that a local variable has the same value on all processors. More...
template<typename T >
bool	semiverify (const T *r) const
	Verify that a local pointer points to the same value on all processors where it is not nullptr. More...
template<typename T >
void	min (const T &r, T &o, Request &req) const
	Non-blocking minimum of the local value r into o with the request req. More...
template<typename T >
void	min (T &r) const
	Take a local variable and replace it with the minimum of it's values on all processors. More...
template<typename T >
void	minloc (T &r, unsigned int &min_id) const
	Take a local variable and replace it with the minimum of it's values on all processors, returning the minimum rank of a processor which originally held the minimum value. More...
template<typename T , typename A1 , typename A2 >
void	minloc (std::vector< T, A1 > &r, std::vector< unsigned int, A2 > &min_id) const
	Take a vector of local variables and replace each entry with the minimum of it's values on all processors. More...
template<typename T >
void	max (const T &r, T &o, Request &req) const
	Non-blocking maximum of the local value r into o with the request req. More...
template<typename T >
void	max (T &r) const
	Take a local variable and replace it with the maximum of it's values on all processors. More...
template<typename T >
void	maxloc (T &r, unsigned int &max_id) const
	Take a local variable and replace it with the maximum of it's values on all processors, returning the minimum rank of a processor which originally held the maximum value. More...
template<typename T , typename A1 , typename A2 >
void	maxloc (std::vector< T, A1 > &r, std::vector< unsigned int, A2 > &max_id) const
	Take a vector of local variables and replace each entry with the maximum of it's values on all processors. More...
template<typename T >
void	sum (T &r) const
	Take a local variable and replace it with the sum of it's values on all processors. More...
template<typename T >
void	sum (const T &r, T &o, Request &req) const
	Non-blocking sum of the local value r into o with the request req. More...
template<typename T >
void	set_union (T &data, const unsigned int root_id) const
	Take a container (set, map, unordered_set, multimap, etc) of local variables on each processor, and collect their union over all processors, replacing the original on processor 0. More...
template<typename T >
void	set_union (T &data) const
	Take a container of local variables on each processor, and replace it with their union over all processors, replacing the original on all processors. More...
status	probe (const unsigned int src_processor_id, const MessageTag &tag=any_tag) const
	Blocking message probe. More...
template<typename T >
Status	packed_range_probe (const unsigned int src_processor_id, const MessageTag &tag, bool &flag) const
	Non-Blocking message probe for a packed range message. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const T &buf, const MessageTag &tag=no_tag) const
	Blocking-send to one processor with data-defined type. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const T &buf, Request &req, const MessageTag &tag=no_tag) const
	Nonblocking-send to one processor with data-defined type. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const T &buf, const DataType &type, const MessageTag &tag=no_tag) const
	Blocking-send to one processor with user-defined type. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const T &buf, const DataType &type, Request &req, const MessageTag &tag=no_tag) const
	Nonblocking-send to one processor with user-defined type. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const T &buf, const NotADataType &type, Request &req, const MessageTag &tag=no_tag) const
	Nonblocking-send to one processor with user-defined packable type. More...
template<typename T >
Status	receive (const unsigned int dest_processor_id, T &buf, const MessageTag &tag=any_tag) const
	Blocking-receive from one processor with data-defined type. More...
template<typename T >
void	receive (const unsigned int dest_processor_id, T &buf, Request &req, const MessageTag &tag=any_tag) const
	Nonblocking-receive from one processor with data-defined type. More...
template<typename T >
Status	receive (const unsigned int dest_processor_id, T &buf, const DataType &type, const MessageTag &tag=any_tag) const
	Blocking-receive from one processor with user-defined type. More...
template<typename T >
void	receive (const unsigned int dest_processor_id, T &buf, const DataType &type, Request &req, const MessageTag &tag=any_tag) const
	Nonblocking-receive from one processor with user-defined type. More...
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
bool	possibly_receive (unsigned int &src_processor_id, std::vector< T, A > &buf, Request &req, const MessageTag &tag) const
	Nonblocking-receive from one processor with user-defined type. More...
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
bool	possibly_receive (unsigned int &src_processor_id, std::vector< T, A > &buf, Request &req, const MessageTag &tag) const
	dispatches to possibly_receive_packed_range More...
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
bool	possibly_receive (unsigned int &src_processor_id, std::vector< T, A > &buf, const DataType &type, Request &req, const MessageTag &tag) const
	Nonblocking-receive from one processor with user-defined type. More...
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
bool	possibly_receive (unsigned int &src_processor_id, std::vector< T, A > &buf, const NotADataType &type, Request &req, const MessageTag &tag) const
	Nonblocking-receive from one processor with user-defined type. More...
template<typename Context , typename OutputIter , typename T >
bool	possibly_receive_packed_range (unsigned int &src_processor_id, Context *context, OutputIter out, const T *output_type, Request &req, const MessageTag &tag) const
	Nonblocking packed range receive from one processor with user-defined type. More...
template<typename Context , typename Iter >
void	send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, const MessageTag &tag=no_tag, std::size_t approx_buffer_size=1000000) const
	Blocking-send range-of-pointers to one processor. More...
template<typename Context , typename Iter >
void	send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, Request &req, const MessageTag &tag=no_tag, std::size_t approx_buffer_size=1000000) const
	Nonblocking-send range-of-pointers to one processor. More...
template<typename Context , typename Iter >
void	nonblocking_send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, Request &req, const MessageTag &tag=no_tag) const
	Similar to the above Nonblocking send_packed_range with a few important differences: More...
template<typename Context , typename Iter >
void	nonblocking_send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, Request &req, std::shared_ptr< std::vector< typename TIMPI::Packing< typename std::iterator_traits< Iter >::value_type >::buffer_type >> &buffer, const MessageTag &tag=no_tag) const
	Similar to the above Nonblocking send_packed_range with a few important differences: More...
template<typename Context , typename OutputIter , typename T >
void	receive_packed_range (const unsigned int dest_processor_id, Context *context, OutputIter out, const T *output_type, const MessageTag &tag=any_tag) const
	Blocking-receive range-of-pointers from one processor. More...
template<typename Context , typename OutputIter , typename T >
void	nonblocking_receive_packed_range (const unsigned int src_processor_id, Context *context, OutputIter out, const T *output_type, Request &req, Status &stat, const MessageTag &tag=any_tag) const
	Non-Blocking-receive range-of-pointers from one processor. More...
template<typename Context , typename OutputIter , typename T >
void	nonblocking_receive_packed_range (const unsigned int src_processor_id, Context *context, OutputIter out, const T *output_type, Request &req, Status &stat, std::shared_ptr< std::vector< typename TIMPI::Packing< T >::buffer_type >> &buffer, const MessageTag &tag=any_tag) const
	Non-Blocking-receive range-of-pointers from one processor. More...
template<typename T1 , typename T2 , typename std::enable_if< std::is_base_of< DataType, StandardType< T1 >>::value &&std::is_base_of< DataType, StandardType< T2 >>::value, int >::type = 0>
void	send_receive (const unsigned int dest_processor_id, const T1 &send_data, const unsigned int source_processor_id, T2 &recv_data, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag) const
	Send data send to one processor while simultaneously receiving other data recv from a (potentially different) processor. More...
template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >
void	send_receive_packed_range (const unsigned int dest_processor_id, const Context1 *context1, RangeIter send_begin, const RangeIter send_end, const unsigned int source_processor_id, Context2 *context2, OutputIter out, const T *output_type, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag, std::size_t approx_buffer_size=1000000) const
	Send a range-of-pointers to one processor while simultaneously receiving another range from a (potentially different) processor. More...
template<typename T1 , typename T2 >
void	send_receive (const unsigned int dest_processor_id, const T1 &send_data, const DataType &type1, const unsigned int source_processor_id, T2 &recv_data, const DataType &type2, const MessageTag &send_tag=no_tag, const MessageTag &recv_tag=any_tag) const
	Send data send to one processor while simultaneously receiving other data recv from a (potentially different) processor, using a user-specified MPI Dataype. More...
template<typename T , typename A >
void	gather (const unsigned int root_id, const T &send_data, std::vector< T, A > &recv) const
	Take a vector of length comm.size(), and on processor root_id fill in recv[processor_id] = the value of send on processor processor_id. More...
template<typename T , typename A >
void	gather (const unsigned int root_id, const std::basic_string< T > &send_data, std::vector< std::basic_string< T >, A > &recv_data, const bool identical_buffer_sizes=false) const
	The gather overload for string types has an optional identical_buffer_sizes optimization for when all strings are the same length. More...
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
void	gather (const unsigned int root_id, std::vector< T, A > &r) const
	Take a vector of local variables and expand it on processor root_id to include values from all processors More...
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
void	gather (const unsigned int root_id, std::vector< T, A > &r) const
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
void	allgather (const T &send_data, std::vector< T, A > &recv_data) const
	Take a vector of length this->size(), and fill in recv[processor_id] = the value of send on that processor. More...
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
void	allgather (const T &send_data, std::vector< T, A > &recv_data) const
	Take a vector of length this->size(), and fill in recv[processor_id] = the value of send on that processor. More...
template<typename T , typename A >
void	allgather (const std::basic_string< T > &send_data, std::vector< std::basic_string< T >, A > &recv_data, const bool identical_buffer_sizes=false) const
	The allgather overload for string types has an optional identical_buffer_sizes optimization for when all strings are the same length. More...
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
void	allgather (std::vector< T, A > &r, const bool identical_buffer_sizes=false) const
	Take a vector of fixed size local variables and expand it to include values from all processors. More...
template<typename T , typename A1 , typename A2 , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0>
void	allgather (const std::vector< T, A1 > &send_data, std::vector< std::vector< T, A1 >, A2 > &recv_data, const bool identical_buffer_sizes=false) const
	Take a vector of fixed size local variables and collect similar vectors from all processors. More...
template<typename T , typename A1 , typename A2 , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
void	allgather (const std::vector< T, A1 > &send_data, std::vector< std::vector< T, A1 >, A2 > &recv_data, const bool identical_buffer_sizes=false) const
	Take a vector of dynamic-size local variables and collect similar vectors from all processors. More...
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
void	allgather (std::vector< T, A > &r, const bool identical_buffer_sizes=false) const
	Take a vector of possibly dynamically sized local variables and expand it to include values from all processors. More...
template<typename T , typename A >
void	allgather (std::vector< std::basic_string< T >, A > &r, const bool identical_buffer_sizes=false) const
	AllGather overload for vectors of string types. More...
template<typename T , typename A >
void	scatter (const std::vector< T, A > &data, T &recv, const unsigned int root_id=0) const
	Take a vector of local variables and scatter the ith item to the ith processor in the communicator. More...
template<typename T , typename A >
void	scatter (const std::vector< T, A > &data, std::vector< T, A > &recv, const unsigned int root_id=0) const
	Take a vector of local variables and scatter the ith equal-sized chunk to the ith processor in the communicator. More...
template<typename T , typename A1 , typename A2 >
void	scatter (const std::vector< T, A1 > &data, const std::vector< int, A2 > counts, std::vector< T, A1 > &recv, const unsigned int root_id=0) const
	Take a vector of local variables and scatter the ith variable-sized chunk to the ith processor in the communicator. More...
template<typename T , typename A1 , typename A2 >
void	scatter (const std::vector< std::vector< T, A1 >, A2 > &data, std::vector< T, A1 > &recv, const unsigned int root_id=0, const bool identical_buffer_sizes=false) const
	Take a vector of vectors and scatter the ith inner vector to the ith processor in the communicator. More...
template<typename Context , typename Iter , typename OutputIter >
void	gather_packed_range (const unsigned int root_id, Context *context, Iter range_begin, const Iter range_end, OutputIter out, std::size_t approx_buffer_size=1000000) const
	Take a range of local variables, combine it with ranges from all processors, and write the output to the output iterator on rank root. More...
template<typename Context , typename Iter , typename OutputIter >
void	allgather_packed_range (Context *context, Iter range_begin, const Iter range_end, OutputIter out, std::size_t approx_buffer_size=1000000) const
	Take a range of local variables, combine it with ranges from all processors, and write the output to the output iterator. More...
template<typename T , typename A >
void	alltoall (std::vector< T, A > &r) const
	Effectively transposes the input vector across all processors. More...
template<typename T #ifdef TIMPI_HAVE_MPI, typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type = 0 #endif>
void	broadcast (T &data, const unsigned int root_id=0, const bool identical_sizes=false) const
	Take a local value and broadcast it to all processors. More...
template<typename T , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>
void	broadcast (T &data, const unsigned int root_id=0, const bool identical_sizes=false) const
	Take a possibly dynamically-sized local value and broadcast it to all processors. More...
template<typename Context , typename OutputContext , typename Iter , typename OutputIter >
void	broadcast_packed_range (const Context *context1, Iter range_begin, const Iter range_end, OutputContext *context2, OutputIter out, const unsigned int root_id=0, std::size_t approx_buffer_size=1000000) const
	Blocking-broadcast range-of-pointers to one processor. More...
template<typename T >
void	send (const unsigned int dest_processor_id, const std::basic_string< T > &buf, const MessageTag &tag) const
template<typename T >
void	send (const unsigned int dest_processor_id, const std::basic_string< T > &buf, Request &req, const MessageTag &tag) const
template<typename T >
Status	receive (const unsigned int src_processor_id, std::basic_string< T > &buf, const MessageTag &tag) const
template<typename T >
void	receive (const unsigned int src_processor_id, std::basic_string< T > &buf, Request &req, const MessageTag &tag) const
template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >
void	send_receive (const unsigned int dest_processor_id, const std::vector< T, A > &send_data, const unsigned int source_processor_id, std::vector< T, A > &recv_data, const MessageTag &send_tag, const MessageTag &recv_tag) const
template<typename T , typename A1 , typename A2 >
void	send_receive (const unsigned int dest_processor_id, const std::vector< std::vector< T, A1 >, A2 > &sendvec, const unsigned int source_processor_id, std::vector< std::vector< T, A1 >, A2 > &recv, const MessageTag &send_tag, const MessageTag &recv_tag) const
template<typename Context , typename Iter >
void	nonblocking_send_packed_range (const unsigned int dest_processor_id, const Context *context, Iter range_begin, const Iter range_end, Request &req, std::shared_ptr< std::vector< typename Packing< typename std::iterator_traits< Iter >::value_type >::buffer_type >> &buffer, const MessageTag &tag) const
template<typename T >
void	broadcast (std::basic_string< T > &data, const unsigned int root_id, const bool identical_sizes) const
template<typename T , typename A >
void	broadcast (std::vector< std::basic_string< T >, A > &data, const unsigned int root_id, const bool identical_sizes) const
template<typename Context , typename OutputIter , typename T >
void	nonblocking_receive_packed_range (const unsigned int src_processor_id, Context *context, OutputIter out, const T *, Request &req, Status &stat, std::shared_ptr< std::vector< typename Packing< T >::buffer_type >> &buffer, const MessageTag &tag) const
template<typename T , typename A1 , typename A2 >
bool	possibly_receive (unsigned int &src_processor_id, std::vector< std::vector< T, A1 >, A2 > &buf, const DataType &type, Request &req, const MessageTag &tag) const
template<typename T >
void	min (T &timpi_mpi_var(r)) const
template<typename A >
void	min (std::vector< bool, A > &r) const
template<typename A1 , typename A2 >
void	minloc (std::vector< bool, A1 > &r, std::vector< unsigned int, A2 > &min_id) const
template<typename T >
void	max (T &timpi_mpi_var(r)) const
template<typename A >
void	max (std::vector< bool, A > &r) const
template<typename A1 , typename A2 >
void	maxloc (std::vector< bool, A1 > &r, std::vector< unsigned int, A2 > &max_id) const
template<typename T >
void	sum (T &timpi_mpi_var(r)) const
template<typename T >
void	sum (std::complex< T > &timpi_mpi_var(r)) const
template<typename T #ifdef TIMPI_HAVE_MPI, typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type # endif>
void	broadcast (T &timpi_mpi_var(data), const unsigned int root_id, const bool) const
template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type >
void	map_broadcast (Map &timpi_mpi_var(data), const unsigned int root_id, const bool timpi_mpi_var(identical_sizes)) const
template<typename Map >
void	map_broadcast (Map &timpi_mpi_var(data), const unsigned int root_id, const bool timpi_mpi_var(identical_sizes)) const
template<typename T , typename A1 , typename A2 >
bool	possibly_receive (unsigned int &src_processor_id, std::vector< std::vector< T, A1 >, A2 > &buf, Request &req, const MessageTag &tag) const
template<typename T1 , typename T2 , typename std::enable_if< std::is_base_of< DataType, StandardType< T1 >>::value &&std::is_base_of< DataType, StandardType< T2 >>::value, int >::type >
void	send_receive (const unsigned int timpi_dbg_var(send_tgt), const T1 &send_val, const unsigned int timpi_dbg_var(recv_source), T2 &recv_val, const MessageTag &, const MessageTag &) const
	Send-receive data from one processor. More...
template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type >
void	send_receive (const unsigned int timpi_dbg_var(dest_processor_id), const std::vector< T, A > &send, const unsigned int timpi_dbg_var(source_processor_id), std::vector< T, A > &recv, const MessageTag &, const MessageTag &) const
template<typename T , typename A1 , typename A2 >
void	send_receive (const unsigned int timpi_dbg_var(dest_processor_id), const std::vector< std::vector< T, A1 >, A2 > &send, const unsigned int timpi_dbg_var(source_processor_id), std::vector< std::vector< T, A1 >, A2 > &recv, const MessageTag &, const MessageTag &) const
template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >
void	send_receive_packed_range (const unsigned int timpi_dbg_var(dest_processor_id), const Context1 *context1, RangeIter send_begin, const RangeIter send_end, const unsigned int timpi_dbg_var(source_processor_id), Context2 *context2, OutputIter out_iter, const T *output_type, const MessageTag &, const MessageTag &, std::size_t) const
	Send-receive range-of-pointers from one processor. More...

Private Member Functions
void	assign (const communicator &comm)
	Utility function for setting our member variables from an MPI communicator. More...
template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type = 0>
void	map_sum (Map &data) const
	Private implementation function called by the map-based sum() specializations. More...
template<typename Map >
void	map_sum (Map &data) const
	Private implementation function called by the map-based sum() specializations. More...
template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type = 0>
void	map_broadcast (Map &data, const unsigned int root_id, const bool identical_sizes) const
	Private implementation function called by the map-based broadcast() specializations. More...
template<typename Map >
void	map_broadcast (Map &data, const unsigned int root_id, const bool identical_sizes) const
	Private implementation function called by the map-based broadcast() specializations. More...
template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type = 0>
void	map_max (Map &data) const
	Private implementation function called by the map-based max() specializations. More...
template<typename Map >
void	map_max (Map &data) const
	Private implementation function called by the map-based max() specializations. More...
template<typename T , typename A1 , typename A2 >
int	packed_size_of (const std::vector< std::vector< T, A1 >, A2 > &buf, const DataType &type) const

Private Attributes
communicator	_communicator
processor_id_type	_rank
processor_id_type	_size
SendMode	_send_mode
SyncType	_sync_type
std::map< int, unsigned int >	used_tag_values
int	_next_tag
int	_max_tag
bool	_I_duped_it

Detailed Description

Encapsulates the MPI_Comm object.

Allows the size of the group and this process's position in the group to be determined.

Methods of this object are the preferred way to perform distributed-memory parallel operations.

Definition at line 106 of file communicator.h.

Member Enumeration Documentation

SendMode

enum TIMPI::Communicator::SendMode

Whether to use default or synchronous sends?

Enumerator
DEFAULT
SYNCHRONOUS

Definition at line 213 of file communicator.h.

{ DEFAULT=0, SYNCHRONOUS };

SyncType

enum TIMPI::Communicator::SyncType

What algorithm to use for parallel synchronization?

Enumerator
NBX
ALLTOALL_COUNTS
SENDRECEIVE

Definition at line 218 of file communicator.h.

{ NBX, ALLTOALL_COUNTS, SENDRECEIVE };

Constructor & Destructor Documentation

Communicator() [1/4]

TIMPI::Communicator::Communicator

(

)

Default Constructor.

Definition at line 59 of file communicator.C.

                            :
#ifdef TIMPI_HAVE_MPI
  _communicator(MPI_COMM_SELF),
#endif
  _rank(0),
  _size(1),
  _send_mode(DEFAULT),
  _sync_type(NBX),
  used_tag_values(),
  _next_tag(0),
  _max_tag(std::numeric_limits<int>::max()),
  _I_duped_it(false) {}

Communicator() [2/4]

TIMPI::Communicator::Communicator ( const communicator &  comm )

explicit

Definition at line 73 of file communicator.C.

References assign().

                                                     :
#ifdef TIMPI_HAVE_MPI
  _communicator(MPI_COMM_SELF),
#endif
  _rank(0),
  _size(1),
  _send_mode(DEFAULT),
  _sync_type(NBX),
  used_tag_values(),
  _next_tag(0),
  _max_tag(std::numeric_limits<int>::max()),
  _I_duped_it(false)
{
  this->assign(comm);
}

Communicator() [3/4]

TIMPI::Communicator::Communicator ( const Communicator &  )

delete

Communicator() [4/4]

TIMPI::Communicator::Communicator ( Communicator &&  )

default

~Communicator()

TIMPI::Communicator::~Communicator

(

)

Definition at line 90 of file communicator.C.

References clear().

{
  this->clear();
}

Member Function Documentation

allgather() [1/8]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::allgather ( const T &  send_data, std::vector< T, A > &  recv_data  ) const

inline

Take a vector of length this->size(), and fill in recv[processor_id] = the value of send on that processor.

This overload works on fixed size types

Definition at line 3193 of file parallel_implementation.h.

References size().

Referenced by allgather(), allgather_packed_range(), gather(), map_max(), map_sum(), testAllGather(), testAllGatherEmptyVectorString(), testAllGatherHalfEmptyVectorString(), testAllGatherString(), testAllGatherVectorString(), testAllGatherVectorVector(), testAllGatherVectorVectorPacked(), testArrayContainerAllGather(), testContainerAllGather(), testMapContainerAllGather(), testPairContainerAllGather(), testTupleContainerAllGather(), and testVectorOfContainersAllGather().

{
  TIMPI_LOG_SCOPE ("allgather()","Parallel");

  timpi_assert(this->size());
  recv.resize(this->size());

  unsigned int comm_size = this->size();
  if (comm_size > 1)
    {
      StandardType<T> send_type(&sendval);

      timpi_call_mpi
        (MPI_Allgather (const_cast<T*>(&sendval), 1, send_type, recv.data(), 1,
                        send_type, this->get()));
    }
  else if (comm_size > 0)
    recv[0] = sendval;
}

allgather() [2/8]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

void TIMPI::Communicator::allgather ( const T &  send_data, std::vector< T, A > &  recv_data  ) const

inline

Take a vector of length this->size(), and fill in recv[processor_id] = the value of send on that processor.

This overload works on potentially dynamically sized types, and dispatches to allgather_packed_range

allgather() [3/8]

template<typename T , typename A >

void TIMPI::Communicator::allgather ( const std::basic_string< T > &  send_data, std::vector< std::basic_string< T >, A > &  recv_data, const bool  identical_buffer_sizes = false  ) const

inline

The allgather overload for string types has an optional identical_buffer_sizes optimization for when all strings are the same length.

Definition at line 1586 of file parallel_implementation.h.

References allgather(), and size().

{
  TIMPI_LOG_SCOPE ("allgather()","Parallel");

  timpi_assert(this->size());
  recv.assign(this->size(), "");

  // serial case
  if (this->size() < 2)
    {
      recv.resize(1);
      recv[0] = sendval;
      return;
    }

  std::vector<int>
    sendlengths  (this->size(), 0),
    displacements(this->size(), 0);

  const int mysize = static_cast<int>(sendval.size());

  if (identical_buffer_sizes)
    sendlengths.assign(this->size(), mysize);
  else
    // first comm step to determine buffer sizes from all processors
    this->allgather(mysize, sendlengths);

  // Find the total size of the final array and
  // set up the displacement offsets for each processor
  unsigned int globalsize = 0;
  for (unsigned int i=0; i != this->size(); ++i)
    {
      displacements[i] = globalsize;
      globalsize += sendlengths[i];
    }

  // Check for quick return
  if (globalsize == 0)
    return;

  // monolithic receive buffer
  std::basic_string<T> r(globalsize, 0);

  // and get the data from the remote processors.
  timpi_call_mpi
    (MPI_Allgatherv (const_cast<T*>(mysize ? sendval.data() : nullptr),
                     mysize, StandardType<T>(),
                     &r[0], sendlengths.data(), displacements.data(),
                     StandardType<T>(), this->get()));

  // slice receive buffer up
  for (unsigned int i=0; i != this->size(); ++i)
    recv[i] = r.substr(displacements[i], sendlengths[i]);
}

allgather() [4/8]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::allgather ( std::vector< T, A > &  r, const bool  identical_buffer_sizes = false  ) const

inline

Take a vector of fixed size local variables and expand it to include values from all processors.

By default, each processor is allowed to have its own unique input buffer length. If it is known that all processors have the same input sizes additional communication can be avoided.

Specifically, this function transforms this:

* Processor 0: [ ... N_0 ]
* Processor 1: [ ....... N_1 ]
* ...
* Processor M: [ .. N_M]
* 

into this:

* [ [ ... N_0 ] [ ....... N_1 ] ... [ .. N_M] ]
* 

on each processor. This function is collective and therefore must be called by all processors in the Communicator.

Definition at line 3242 of file parallel_implementation.h.

References allgather(), size(), and verify().

{
  if (this->size() < 2)
    return;

  TIMPI_LOG_SCOPE("allgather()", "Parallel");

  if (identical_buffer_sizes)
    {
      timpi_assert(this->verify(r.size()));
      if (r.empty())
        return;

      std::vector<T,A> r_src(r.size()*this->size());
      r_src.swap(r);
      StandardType<T> send_type(r_src.data());

      timpi_call_mpi
        (MPI_Allgather (r_src.data(), cast_int<int>(r_src.size()),
                        send_type, r.data(), cast_int<int>(r_src.size()),
                        send_type, this->get()));
      // timpi_assert(this->verify(r));
      return;
    }

  std::vector<int>
    sendlengths  (this->size(), 0),
    displacements(this->size(), 0);

  const int mysize = static_cast<int>(r.size());
  this->allgather(mysize, sendlengths);

  // Find the total size of the final array and
  // set up the displacement offsets for each processor.
  unsigned int globalsize = 0;
  for (unsigned int i=0; i != this->size(); ++i)
    {
      displacements[i] = globalsize;
      globalsize += sendlengths[i];
    }

  // Check for quick return
  if (globalsize == 0)
    return;

  // copy the input buffer
  std::vector<T,A> r_src(globalsize);
  r_src.swap(r);

  StandardType<T> send_type(r.data());

  // and get the data from the remote processors.
  // Pass nullptr if our vector is empty.
  timpi_call_mpi
    (MPI_Allgatherv (r_src.empty() ? nullptr : r_src.data(), mysize,
                     send_type, r.data(), sendlengths.data(),
                     displacements.data(), send_type, this->get()));
}

allgather() [5/8]

template<typename T , typename A1 , typename A2 , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::allgather ( const std::vector< T, A1 > &  send_data, std::vector< std::vector< T, A1 >, A2 > &  recv_data, const bool  identical_buffer_sizes = false  ) const

inline

Take a vector of fixed size local variables and collect similar vectors from all processors.

By default, each processor is allowed to have its own unique input buffer length. If it is known that all processors have the same input sizes additional communication can be avoided.

Definition at line 2804 of file parallel_implementation.h.

References allgather(), assign(), and size().

{
  TIMPI_LOG_SCOPE ("allgather()","Parallel");

  timpi_assert(this->size());

  // serial case
  if (this->size() < 2)
    {
      recv.resize(1);
      recv[0] = sendval;
      return;
    }

  recv.clear();
  recv.resize(this->size());

  std::vector<int>
    sendlengths  (this->size(), 0),
    displacements(this->size(), 0);

  const int mysize = static_cast<int>(sendval.size());

  if (identical_buffer_sizes)
    sendlengths.assign(this->size(), mysize);
  else
    // first comm step to determine buffer sizes from all processors
    this->allgather(mysize, sendlengths);

  // Find the total size of the final array and
  // set up the displacement offsets for each processor
  unsigned int globalsize = 0;
  for (unsigned int i=0; i != this->size(); ++i)
    {
      displacements[i] = globalsize;
      globalsize += sendlengths[i];
    }

  // Check for quick return
  if (globalsize == 0)
    return;

  // monolithic receive buffer
  std::vector<T,A1> r(globalsize, 0);

  // and get the data from the remote processors.
  timpi_call_mpi
    (MPI_Allgatherv (const_cast<T*>(mysize ? sendval.data() : nullptr),
                     mysize, StandardType<T>(),
                     &r[0], sendlengths.data(), displacements.data(),
                     StandardType<T>(), this->get()));

  // slice receive buffer up
  for (unsigned int i=0; i != this->size(); ++i)
    recv[i].assign(r.begin()+displacements[i],
                   r.begin()+displacements[i]+sendlengths[i]);
}

allgather() [6/8]

template<typename T , typename A1 , typename A2 , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

void TIMPI::Communicator::allgather ( const std::vector< T, A1 > &  send_data, std::vector< std::vector< T, A1 >, A2 > &  recv_data, const bool  identical_buffer_sizes = false  ) const

inline

Take a vector of dynamic-size local variables and collect similar vectors from all processors.

allgather() [7/8]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

void TIMPI::Communicator::allgather ( std::vector< T, A > &  r, const bool  identical_buffer_sizes = false  ) const

inline

Take a vector of possibly dynamically sized local variables and expand it to include values from all processors.

By default, each processor is allowed to have its own unique input buffer length. If it is known that all processors have the same input sizes additional communication can be avoided.

Specifically, this function transforms this:

* Processor 0: [ ... N_0 ]
* Processor 1: [ ....... N_1 ]
* ...
* Processor M: [ .. N_M]
* 

into this:

* [ [ ... N_0 ] [ ....... N_1 ] ... [ .. N_M] ]
* 

on each processor. This function is collective and therefore must be called by all processors in the Communicator.

allgather() [8/8]

template<typename T , typename A >

void TIMPI::Communicator::allgather ( std::vector< std::basic_string< T >, A > &  r, const bool  identical_buffer_sizes = false  ) const

inline

AllGather overload for vectors of string types.

Definition at line 3356 of file parallel_implementation.h.

References allgather(), size(), and verify().

{
  if (this->size() < 2)
    return;

  TIMPI_LOG_SCOPE("allgather()", "Parallel");

  if (identical_buffer_sizes)
    {
      timpi_assert(this->verify(r.size()));

      // identical_buffer_sizes doesn't buy us much since we have to
      // communicate the lengths of strings within each buffer anyway
      if (r.empty())
        return;
    }

  // Concatenate the input buffer into a send buffer, and keep track
  // of input string lengths
  std::vector<int> mystrlengths (r.size());
  std::vector<T> concat_src;

  int myconcatsize = 0;
  for (unsigned int i=0; i != r.size(); ++i)
    {
      int stringlen = cast_int<int>(r[i].size());
      mystrlengths[i] = stringlen;
      myconcatsize += stringlen;
    }
  concat_src.reserve(myconcatsize);
  for (unsigned int i=0; i != r.size(); ++i)
    concat_src.insert
      (concat_src.end(), r[i].begin(), r[i].end());

  // Get the string lengths from all other processors
  std::vector<int> strlengths = mystrlengths;
  this->allgather(strlengths, identical_buffer_sizes);

  // We now know how many strings we'll be receiving
  r.resize(strlengths.size());

  // Get the concatenated data sizes from all other processors
  std::vector<int> concat_sizes;
  this->allgather(myconcatsize, concat_sizes);

  // Find the total size of the final concatenated array and
  // set up the displacement offsets for each processor.
  std::vector<int> displacements(this->size(), 0);
  unsigned int globalsize = 0;
  for (unsigned int i=0; i != this->size(); ++i)
    {
      displacements[i] = globalsize;
      globalsize += concat_sizes[i];
    }

  // Check for quick return
  if (globalsize == 0)
    return;

  // Get the concatenated data from the remote processors.
  // Pass nullptr if our vector is empty.
  std::vector<T> concat(globalsize);

  // We may have concat_src.empty(), but we know concat has at least
  // one element we can use as an example for StandardType
  StandardType<T> send_type(concat.data());

  timpi_call_mpi
    (MPI_Allgatherv (concat_src.empty() ?
                     nullptr : concat_src.data(), myconcatsize,
                     send_type, concat.data(), concat_sizes.data(),
                     displacements.data(), send_type, this->get()));

  // Finally, split concatenated data into strings
  const T * begin = concat.data();
  for (unsigned int i=0; i != r.size(); ++i)
    {
      const T * end = begin + strlengths[i];
      r[i].assign(begin, end);
      begin = end;
    }
}

allgather_packed_range()

template<typename Context , typename Iter , typename OutputIter >

void TIMPI::Communicator::allgather_packed_range ( Context *  context, Iter  range_begin, const Iter  range_end, OutputIter  out, std::size_t  approx_buffer_size = 1000000  ) const

inline

Take a range of local variables, combine it with ranges from all processors, and write the output to the output iterator.

The approximate maximum size (in entries; number of bytes will likely be 4x or 8x larger) to use in a single data vector buffer to send can be specified for performance or memory usage reasons; if the range cannot be packed into a single buffer of this size then multiple buffers and messages will be used.

Note that the received data vector sizes will be the sum of the sent vector sizes; a smaller-than-default size may be useful for users on many processors, in cases where all-to-one communication cannot be avoided entirely.

Definition at line 4004 of file parallel_implementation.h.

References allgather(), max(), TIMPI::pack_range(), and TIMPI::unpack_range().

Referenced by testGettableStringAllGather(), testNestingAllGather(), testNullAllGather(), and testTupleStringAllGather().

{
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  bool nonempty_range = (range_begin != range_end);
  this->max(nonempty_range);

  while (nonempty_range)
    {
      // We will serialize variable size objects from *range_begin to
      // *range_end as a sequence of ints in this buffer
      std::vector<buffer_t> buffer;

      range_begin = pack_range
        (context, range_begin, range_end, buffer, approx_buffer_size);

      this->allgather(buffer, false);

      timpi_assert(buffer.size());

      unpack_range
        (buffer, context, out_iter, (T*)nullptr);

      nonempty_range = (range_begin != range_end);
      this->max(nonempty_range);
    }
}

alltoall()

template<typename T , typename A >

void TIMPI::Communicator::alltoall ( std::vector< T, A > &  r ) const

inline

Effectively transposes the input vector across all processors.

The jth entry on processor i is replaced with the ith entry from processor j.

Definition at line 3620 of file parallel_implementation.h.

References TIMPI::ignore(), size(), and verify().

Referenced by TIMPI::detail::push_parallel_alltoall_helper().

{
  if (this->size() < 2 || buf.empty())
    return;

  TIMPI_LOG_SCOPE("alltoall()", "Parallel");

  // the per-processor size.  this is the same for all
  // processors using MPI_Alltoall, could be variable
  // using MPI_Alltoallv
  const int size_per_proc =
    cast_int<int>(buf.size()/this->size());
  ignore(size_per_proc);

  timpi_assert_equal_to (buf.size()%this->size(), 0);

  timpi_assert(this->verify(size_per_proc));

  StandardType<T> send_type(buf.data());

  timpi_call_mpi
    (MPI_Alltoall (MPI_IN_PLACE, size_per_proc, send_type, buf.data(),
                   size_per_proc, send_type, this->get()));
}

assign()

void TIMPI::Communicator::assign ( const communicator &  comm )

private

Utility function for setting our member variables from an MPI communicator.

Definition at line 185 of file communicator.C.

References _communicator, _max_tag, _next_tag, _rank, and _size.

Referenced by allgather(), Communicator(), duplicate(), operator=(), split(), and split_by_type().

{
  _communicator = comm;
#ifdef TIMPI_HAVE_MPI
  if (_communicator != MPI_COMM_NULL)
    {
      int i;
      timpi_call_mpi
        (MPI_Comm_size(_communicator, &i));

      timpi_assert_greater_equal (i, 0);
      _size = cast_int<processor_id_type>(i);

      timpi_call_mpi
        (MPI_Comm_rank(_communicator, &i));

      timpi_assert_greater_equal (i, 0);
      _rank = cast_int<processor_id_type>(i);

      int * maxTag;
      int flag = false;
      timpi_call_mpi(MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_TAG_UB, &maxTag, &flag));
      timpi_assert(flag);
      _max_tag = *maxTag;
    }
  else
    {
      _rank = 0;
      _size = 1;
      _max_tag = std::numeric_limits<int>::max();
    }
  _next_tag = _max_tag / 2;
#endif
}

barrier()

void TIMPI::Communicator::barrier

(

)

const

Pause execution until all processors reach a certain point.

Definition at line 225 of file communicator.C.

References size().

Referenced by testBarrier(), and TIMPI::TIMPIInit::~TIMPIInit().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("barrier()", "Communicator");
      timpi_call_mpi(MPI_Barrier (this->get()));
    }
}

broadcast() [1/5]

template<typename T , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::broadcast ( T &  data, const unsigned int  root_id = 0, const bool  identical_sizes = false  ) const

inline

Take a local value and broadcast it to all processors.

Optionally takes the root_id processor, which specifies the processor initiating the broadcast.

If data is a container, it will be resized on target processors. When using pre-sized target containers, specify identical_sizes=true on all processors for an optimization.

Fixed variant

Definition at line 3678 of file parallel_implementation.h.

References broadcast_packed_range(), TIMPI::ignore(), rank(), and size().

Referenced by broadcast(), broadcast_packed_range(), map_broadcast(), scatter(), testBroadcast(), testBroadcastArrayType(), testBroadcastNestedType(), testBroadcastString(), testContainerBroadcast(), and testVectorOfContainersBroadcast().

{
  ignore(root_id); // Only needed for MPI and/or dbg/devel
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

  timpi_assert_less (root_id, this->size());

//   // If we don't have MPI, then we should be done, and calling the below can
//   // have the side effect of instantiating Packing<T> classes that are not
//   // defined. (Normally we would be calling a more specialized overload of
//   // broacast that would then call broadcast_packed_range with appropriate
//   // template arguments)
// #ifdef TIMPI_HAVE_MPI
  std::vector<T> range = {data};

  this->broadcast_packed_range((void *)(nullptr),
                               range.begin(),
                               range.end(),
                               (void *)(nullptr),
                               range.begin(),
                               root_id);

  data = range[0];
// #endif
}

broadcast() [2/5]

template<typename T , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

void TIMPI::Communicator::broadcast ( T &  data, const unsigned int  root_id = 0, const bool  identical_sizes = false  ) const

inline

Take a possibly dynamically-sized local value and broadcast it to all processors.

Optionally takes the root_id processor, which specifies the processor initiating the broadcast.

If data is a container, it will be resized on target processors. When using pre-sized target containers, specify identical_sizes=true on all processors for an optimization.

Dynamic variant

broadcast() [3/5]

template<typename T >

void TIMPI::Communicator::broadcast ( std::basic_string< T > &  data, const unsigned int  root_id, const bool  identical_sizes  ) const

inline

Definition at line 1676 of file parallel_implementation.h.

References broadcast(), rank(), size(), and verify().

{
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

  timpi_assert_less (root_id, this->size());
  timpi_assert (this->verify(identical_sizes));

  TIMPI_LOG_SCOPE("broadcast()", "Parallel");

  std::size_t data_size = data.size();

  if (identical_sizes)
    timpi_assert(this->verify(data_size));
  else
    this->broadcast(data_size, root_id);

  std::vector<T> data_c(data_size);
#ifndef NDEBUG
  std::basic_string<T> orig(data);
#endif

  if (this->rank() == root_id)
    for (std::size_t i=0; i<data.size(); i++)
      data_c[i] = data[i];

  this->broadcast (data_c, root_id, StandardType<T>::is_fixed_type);

  data.assign(data_c.begin(), data_c.end());

#ifndef NDEBUG
  if (this->rank() == root_id)
    timpi_assert_equal_to (data, orig);
#endif
}

broadcast() [4/5]

template<typename T , typename A >

void TIMPI::Communicator::broadcast ( std::vector< std::basic_string< T >, A > &  data, const unsigned int  root_id, const bool  identical_sizes  ) const

inline

The strings will be packed in one long array with the size of each string preceding the actual characters

Definition at line 1720 of file parallel_implementation.h.

References broadcast(), rank(), size(), and verify().

{
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

  timpi_assert_less (root_id, this->size());
  timpi_assert (this->verify(identical_sizes));

  TIMPI_LOG_SCOPE("broadcast()", "Parallel");

  std::size_t bufsize=0;
  if (root_id == this->rank() || identical_sizes)
    {
      for (std::size_t i=0; i<data.size(); ++i)
        bufsize += data[i].size() + 1;  // Add one for the string length word
    }

  if (identical_sizes)
    timpi_assert(this->verify(bufsize));
  else
    this->broadcast(bufsize, root_id);

  // Here we use unsigned int to store up to 32-bit characters
  std::vector<unsigned int> temp; temp.reserve(bufsize);
  // Pack the strings
  if (root_id == this->rank())
    {
      for (std::size_t i=0; i<data.size(); ++i)
        {
          temp.push_back(cast_int<unsigned int>(data[i].size()));
          for (std::size_t j=0; j != data[i].size(); ++j)
            temp.push_back(data[i][j]);
        }
    }
  else
    temp.resize(bufsize);

  // broad cast the packed strings
  this->broadcast(temp, root_id, true);

  // Unpack the strings
  if (root_id != this->rank())
    {
      data.clear();
      typename std::vector<unsigned int>::const_iterator iter = temp.begin();
      while (iter != temp.end())
        {
          std::size_t curr_len = *iter++;
          data.push_back(std::basic_string<T>(iter, iter+curr_len));
          iter += curr_len;
        }
    }
}

broadcast() [5/5]

template<typename T #ifdef TIMPI_HAVE_MPI, typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type # endif>

void TIMPI::Communicator::broadcast ( T &  timpi_mpi_vardata, const unsigned int  root_id, const bool    ) const

inline

Definition at line 3653 of file parallel_implementation.h.

References TIMPI::ignore(), rank(), and size().

{
  ignore(root_id); // Only needed for MPI and/or dbg/devel
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

  timpi_assert_less (root_id, this->size());

  TIMPI_LOG_SCOPE("broadcast()", "Parallel");

  // Spread data to remote processors.
  timpi_call_mpi
    (MPI_Bcast (&data, 1, StandardType<T>(&data), root_id,
                this->get()));
}

broadcast_packed_range()

template<typename Context , typename OutputContext , typename Iter , typename OutputIter >

void TIMPI::Communicator::broadcast_packed_range ( const Context *  context1, Iter  range_begin, const Iter  range_end, OutputContext *  context2, OutputIter  out, const unsigned int  root_id = 0, std::size_t  approx_buffer_size = 1000000  ) const

inline

Blocking-broadcast range-of-pointers to one processor.

This function does not send the raw pointers, but rather constructs new objects at the other end whose contents match the objects pointed to by the sender.

void TIMPI::pack(const T *, vector<int> & data, const Context *) is used to serialize type T onto the end of a data vector.

unsigned int TIMPI::packable_size(const T *, const Context *) is used to allow data vectors to reserve memory, and for additional error checking

unsigned int TIMPI::packed_size(const T *, vector<int>::const_iterator) is used to advance to the beginning of the next object's data.

The approximate maximum size (in entries; number of bytes will likely be 4x or 8x larger) to use in a single data vector buffer can be specified for performance or memory usage reasons; if the range cannot be packed into a single buffer of this size then multiple buffers and messages will be used.

Definition at line 3920 of file parallel_implementation.h.

References broadcast(), TIMPI::pack_range(), rank(), size(), and TIMPI::unpack_range().

Referenced by broadcast().

{
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  if (this->size() == 1)
  {
    timpi_assert (!this->rank());
    timpi_assert (!root_id);
    return;
  }

  do
  {
    // We will serialize variable size objects from *range_begin to
    // *range_end as a sequence of ints in this buffer
    std::vector<buffer_t> buffer;

    if (this->rank() == root_id)
      range_begin = pack_range
        (context1, range_begin, range_end, buffer, approx_buffer_size);

    // this->broadcast(vector) requires the receiving vectors to
    // already be the appropriate size
    std::size_t buffer_size = buffer.size();
    this->broadcast (buffer_size, root_id);

    // We continue until there's nothing left to broadcast
    if (!buffer_size)
      break;

    buffer.resize(buffer_size);

    // Broadcast the packed data
    this->broadcast (buffer, root_id);

    if (this->rank() != root_id)
      unpack_range
        (buffer, context2, out_iter, (T*)nullptr);
  } while (true);  // break above when we reach buffer_size==0
}

clear()

void TIMPI::Communicator::clear

(

)

Free and reset this communicator.

Definition at line 162 of file communicator.C.

References _communicator, and _I_duped_it.

Referenced by operator=(), split(), split_by_type(), and ~Communicator().

                         {
#ifdef TIMPI_HAVE_MPI
  if (_I_duped_it)
    {
      timpi_assert (_communicator != MPI_COMM_NULL);
      timpi_call_mpi
        (MPI_Comm_free(&_communicator));

      _communicator = MPI_COMM_NULL;
    }
  _I_duped_it = false;
#endif
}

dereference_unique_tag()

void TIMPI::Communicator::dereference_unique_tag

(

int

tagvalue

)

const

Dereference an already-acquired tag, and see if we can re-release it.

Definition at line 46 of file communicator.C.

References used_tag_values.

Referenced by TIMPI::MessageTag::operator=(), and TIMPI::MessageTag::~MessageTag().

{
  // This had better be an already-acquired tag.
  timpi_assert(used_tag_values.count(tagvalue));

  used_tag_values[tagvalue]--;
  // If we don't have any more outstanding references, we
  // don't even need to keep this tag in our "used" set.
  if (!used_tag_values[tagvalue])
    used_tag_values.erase(tagvalue);
}

duplicate() [1/2]

void TIMPI::Communicator::duplicate

(

const Communicator &

comm

)

Definition at line 137 of file communicator.C.

References _communicator, send_mode(), and sync_type().

Referenced by testGetUniqueTagAuto(), and testStringSyncType().

{
  this->duplicate(comm._communicator);
  this->send_mode(comm.send_mode());
  this->sync_type(comm.sync_type());
}

duplicate() [2/2]

void TIMPI::Communicator::duplicate

(

const communicator &

comm

)

Definition at line 146 of file communicator.C.

References _communicator, _I_duped_it, and assign().

{
  if (_communicator != MPI_COMM_NULL)
    {
      timpi_call_mpi
        (MPI_Comm_dup(comm, &_communicator));

      _I_duped_it = true;
    }
  this->assign(_communicator);
}

gather() [1/4]

template<typename T , typename A >

void TIMPI::Communicator::gather ( const unsigned int  root_id, const T &  send_data, std::vector< T, A > &  recv  ) const

inline

Take a vector of length comm.size(), and on processor root_id fill in recv[processor_id] = the value of send on processor processor_id.

Definition at line 3031 of file parallel_implementation.h.

References rank(), and size().

Referenced by gather(), gather_packed_range(), testGather(), testGatherString(), and testGatherString2().

{
  timpi_assert_less (root_id, this->size());

  if (this->rank() == root_id)
    recv.resize(this->size());

  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("gather()", "Parallel");

      StandardType<T> send_type(&sendval);

      timpi_assert_less(root_id, this->size());

      timpi_call_mpi
        (MPI_Gather(const_cast<T*>(&sendval), 1, send_type,
                    recv.empty() ? nullptr : recv.data(), 1, send_type,
                    root_id, this->get()));
    }
  else
    recv[0] = sendval;
}

gather() [2/4]

template<typename T , typename A >

void TIMPI::Communicator::gather ( const unsigned int  root_id, const std::basic_string< T > &  send_data, std::vector< std::basic_string< T >, A > &  recv_data, const bool  identical_buffer_sizes = false  ) const

inline

The gather overload for string types has an optional identical_buffer_sizes optimization for when all strings are the same length.

Definition at line 3130 of file parallel_implementation.h.

References gather(), rank(), and size().

{
  timpi_assert_less (root_id, this->size());

  if (this->rank() == root_id)
    recv.resize(this->size());

  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE ("gather()","Parallel");

      std::vector<int>
        sendlengths  (this->size(), 0),
        displacements(this->size(), 0);

      const int mysize = static_cast<int>(sendval.size());

      if (identical_buffer_sizes)
        sendlengths.assign(this->size(), mysize);
      else
        // first comm step to determine buffer sizes from all processors
        this->gather(root_id, mysize, sendlengths);

      // Find the total size of the final array and
      // set up the displacement offsets for each processor
      unsigned int globalsize = 0;
      for (unsigned int i=0; i < this->size(); ++i)
        {
          displacements[i] = globalsize;
          globalsize += sendlengths[i];
        }

      // monolithic receive buffer
      std::basic_string<T> r;
      if (this->rank() == root_id)
        r.resize(globalsize, 0);

      timpi_assert_less(root_id, this->size());

      // and get the data from the remote processors.
      timpi_call_mpi
        (MPI_Gatherv (const_cast<T*>(sendval.data()),
                      mysize, StandardType<T>(),
                      this->rank() == root_id ? &r[0] : nullptr,
                      sendlengths.data(), displacements.data(),
                      StandardType<T>(), root_id, this->get()));

      // slice receive buffer up
      if (this->rank() == root_id)
        for (unsigned int i=0; i != this->size(); ++i)
          recv[i] = r.substr(displacements[i], sendlengths[i]);
    }
  else
    recv[0] = sendval;
}

gather() [3/4]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::gather ( const unsigned int  root_id, std::vector< T, A > &  r  ) const

inline

Take a vector of local variables and expand it on processor root_id to include values from all processors

This handles the case where the lengths of the vectors may vary. Specifically, this function transforms this:

* Processor 0: [ ... N_0 ]
* Processor 1: [ ....... N_1 ]
* ...
* Processor M: [ .. N_M]
* 

into this:

* [ [ ... N_0 ] [ ....... N_1 ] ... [ .. N_M] ]
* 

on processor root_id. This function is collective and therefore must be called by all processors in the Communicator.

If the type T is a standard (fixed-size) type then we use a standard MPI_Gatherv; if it is a packable variable-size type then we dispatch to gather_packed_range.

Definition at line 3061 of file parallel_implementation.h.

References allgather(), rank(), and size().

{
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

  timpi_assert_less (root_id, this->size());

  std::vector<int>
    sendlengths  (this->size(), 0),
    displacements(this->size(), 0);

  const int mysize = static_cast<int>(r.size());
  this->allgather(mysize, sendlengths);

  TIMPI_LOG_SCOPE("gather()", "Parallel");

  // Find the total size of the final array and
  // set up the displacement offsets for each processor.
  unsigned int globalsize = 0;
  for (unsigned int i=0; i != this->size(); ++i)
    {
      displacements[i] = globalsize;
      globalsize += sendlengths[i];
    }

  // Check for quick return
  if (globalsize == 0)
    return;

  // copy the input buffer
  std::vector<T,A> r_src(r);

  // now resize it to hold the global data
  // on the receiving processor
  if (root_id == this->rank())
    r.resize(globalsize);

  timpi_assert_less(root_id, this->size());

  // and get the data from the remote processors
  timpi_call_mpi
    (MPI_Gatherv (r_src.empty() ? nullptr : r_src.data(), mysize,
                  StandardType<T>(), r.empty() ? nullptr : r.data(),
                  sendlengths.data(), displacements.data(),
                  StandardType<T>(), root_id, this->get()));
}

gather() [4/4]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

void TIMPI::Communicator::gather ( const unsigned int  root_id, std::vector< T, A > &  r  ) const

inline

gather_packed_range()

template<typename Context , typename Iter , typename OutputIter >

void TIMPI::Communicator::gather_packed_range ( const unsigned int  root_id, Context *  context, Iter  range_begin, const Iter  range_end, OutputIter  out, std::size_t  approx_buffer_size = 1000000  ) const

inline

Take a range of local variables, combine it with ranges from all processors, and write the output to the output iterator on rank root.

The approximate maximum size (in entries; number of bytes will likely be 4x or 8x larger) to use in a single data vector buffer to send can be specified for performance or memory usage reasons; if the range cannot be packed into a single buffer of this size then multiple buffers and messages will be used.

Note that the received data vector sizes will be the sum of the sent vector sizes; a smaller-than-default size may be useful for users on many processors, in cases where all-to-one communication cannot be avoided entirely.

Definition at line 3970 of file parallel_implementation.h.

References gather(), max(), TIMPI::pack_range(), and TIMPI::unpack_range().

{
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  bool nonempty_range = (range_begin != range_end);
  this->max(nonempty_range);

  while (nonempty_range)
    {
      // We will serialize variable size objects from *range_begin to
      // *range_end as a sequence of ints in this buffer
      std::vector<buffer_t> buffer;

      range_begin = pack_range
        (context, range_begin, range_end, buffer, approx_buffer_size);

      this->gather(root_id, buffer);

      unpack_range
        (buffer, context, out_iter, (T*)(nullptr));

      nonempty_range = (range_begin != range_end);
      this->max(nonempty_range);
    }
}

get() [1/2]

communicator& TIMPI::Communicator::get ( )

inline

Definition at line 165 of file communicator.h.

References _communicator.

Referenced by TIMPI::PostWaitUnpackNestedBuffer< Container >::run(), and testMPIULongMin().

{ return _communicator; }

get() [2/2]

const communicator& TIMPI::Communicator::get ( ) const

inline

Definition at line 167 of file communicator.h.

References _communicator.

{ return _communicator; }

get_unique_tag()

MessageTag TIMPI::Communicator::get_unique_tag

(

int

tagvalue = MessageTag::invalid_tag

)

const

Get a tag that is unique to this Communicator.

A requested tag value may be provided. If no request is made then an automatic unique tag value will be generated; such usage of get_unique_tag() must be done on every processor in a consistent order.

Note

If people are also using magic numbers or copying raw communicators around then we can't guarantee the tag is unique to this MPI_Comm.

Leaving tagvalue unspecified is recommended in most cases. Manually selecting tag values is dangerous, as tag values may be freed and reselected earlier than expected in asynchronous communication algorithms.

Definition at line 251 of file communicator.C.

References _max_tag, _next_tag, TIMPI::MessageTag::invalid_tag, max(), and used_tag_values.

Referenced by TIMPI::pull_parallel_vector_data(), TIMPI::detail::push_parallel_alltoall_helper(), TIMPI::detail::push_parallel_nbx_helper(), TIMPI::detail::push_parallel_roundrobin_helper(), testGetUniqueTagAuto(), testGetUniqueTagManual(), and testGetUniqueTagOverlap().

{
  if (tagvalue == MessageTag::invalid_tag)
    {
#ifndef NDEBUG
      // Automatic tag values have to be requested in sync
      int maxval = _next_tag;
      this->max(maxval);
      timpi_assert_equal_to(_next_tag, maxval);
#endif
      tagvalue = _next_tag++;
    }

  if (used_tag_values.count(tagvalue))
    {
      // Get the largest value in the used values, and pick one
      // larger
      tagvalue = used_tag_values.rbegin()->first+1;
      timpi_assert(!used_tag_values.count(tagvalue));
    }
  if (tagvalue >= _next_tag)
    _next_tag = tagvalue+1;

  if (_next_tag >= _max_tag)
    _next_tag = _max_tag/2;

  used_tag_values[tagvalue] = 1;

  return MessageTag(tagvalue, this);
}

map_broadcast() [1/4]

template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type = 0>

void TIMPI::Communicator::map_broadcast ( Map &  data, const unsigned int  root_id, const bool  identical_sizes  ) const

private

Private implementation function called by the map-based broadcast() specializations.

This is_fixed_type variant saves a communication by broadcasting pairs

map_broadcast() [2/4]

template<typename Map >

void TIMPI::Communicator::map_broadcast ( Map &  data, const unsigned int  root_id, const bool  identical_sizes  ) const

private

Private implementation function called by the map-based broadcast() specializations.

This !is_fixed_type variant makes two broadcasts, which is slower but gives more control over reach broadcast (e.g. we may need to specialize for either key_type or mapped_type)

map_broadcast() [3/4]

template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type >

void TIMPI::Communicator::map_broadcast ( Map &  timpi_mpi_vardata, const unsigned int  root_id, const bool   timpi_mpi_varidentical_sizes  ) const

inline

Definition at line 3795 of file parallel_implementation.h.

References broadcast(), TIMPI::ignore(), rank(), size(), and verify().

{
  ignore(root_id); // Only needed for MPI and/or dbg/devel
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

#ifdef TIMPI_HAVE_MPI
  timpi_assert_less (root_id, this->size());
  timpi_assert (this->verify(identical_sizes));

  TIMPI_LOG_SCOPE("broadcast(map)", "Parallel");

  std::size_t data_size=data.size();
  if (identical_sizes)
    timpi_assert(this->verify(data_size));
  else
    this->broadcast(data_size, root_id);

  std::vector<std::pair<typename Map::key_type,
                        typename Map::mapped_type>> comm_data;

  if (root_id == this->rank())
    comm_data.assign(data.begin(), data.end());
  else
    comm_data.resize(data_size);

  this->broadcast(comm_data, root_id, true);

  if (this->rank() != root_id)
    {
      data.clear();
      data.insert(comm_data.begin(), comm_data.end());
    }
#endif
}

map_broadcast() [4/4]

template<typename Map >

void TIMPI::Communicator::map_broadcast ( Map &  timpi_mpi_vardata, const unsigned int  root_id, const bool   timpi_mpi_varidentical_sizes  ) const

inline

Definition at line 3841 of file parallel_implementation.h.

References TIMPI::ignore().

{
  ignore(root_id); // Only needed for MPI and/or dbg/devel
  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      return;
    }

#ifdef TIMPI_HAVE_MPI
  timpi_assert_less (root_id, this->size());
  timpi_assert (this->verify(identical_sizes));

  TIMPI_LOG_SCOPE("broadcast()", "Parallel");

  std::size_t data_size=data.size();
  if (identical_sizes)
    timpi_assert(this->verify(data_size));
  else
    this->broadcast(data_size, root_id);

  std::vector<typename Map::key_type> pair_first; pair_first.reserve(data_size);
  std::vector<typename Map::mapped_type> pair_second; pair_first.reserve(data_size);

  if (root_id == this->rank())
    {
      for (const auto & pr : data)
        {
          pair_first.push_back(pr.first);
          pair_second.push_back(pr.second);
        }
    }
  else
    {
      pair_first.resize(data_size);
      pair_second.resize(data_size);
    }

  this->broadcast
    (pair_first, root_id,
     StandardType<typename Map::key_type>::is_fixed_type);
  this->broadcast
    (pair_second, root_id,
     StandardType<typename Map::mapped_type>::is_fixed_type);

  timpi_assert(pair_first.size() == pair_first.size());

  if (this->rank() != root_id)
    {
      data.clear();
      for (std::size_t i=0; i<pair_first.size(); ++i)
        data[pair_first[i]] = pair_second[i];
    }
#endif
}

map_max() [1/2]

template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type >

void TIMPI::Communicator::map_max ( Map &  data ) const

private

Private implementation function called by the map-based max() specializations.

This is_fixed_type variant saves a communication by broadcasting pairs

Definition at line 2407 of file parallel_implementation.h.

References allgather(), and size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("max(map)", "Parallel");

      // Since the input map may have different keys on different
      // processors, we first gather all the keys and values, then for
      // each key we choose the max value over all procs. We
      // initialize the max with the first value we encounter rather
      // than some "global" minimum, since the latter is difficult to
      // do generically.
      std::vector<std::pair<typename Map::key_type, typename Map::mapped_type>>
        vecdata(data.begin(), data.end());

      this->allgather(vecdata, /*identical_buffer_sizes=*/false);

      data.clear();

      for (const auto & pr : vecdata)
        {
          // Attempt to insert this value. If it works, then the value didn't
          // already exist and we can go on. If it fails, compute the std::max
          // between the current and existing values.
          auto result = data.insert(pr);

          bool inserted = result.second;

          if (!inserted)
            {
              auto it = result.first;
              it->second = std::max(it->second, pr.second);
            }
        }
    }
}

map_max() [2/2]

template<typename Map >

void TIMPI::Communicator::map_max ( Map &  data ) const

private

Private implementation function called by the map-based max() specializations.

This !is_fixed_type variant calls allgather twice: once for the keys and once for the values.

Definition at line 2450 of file parallel_implementation.h.

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("max(map)", "Parallel");

      // Since the input map may have different keys on different
      // processors, we first gather all the keys and values, then for
      // each key we choose the max value over all procs. We
      // initialize the max with the first value we encounter rather
      // than some "global" minimum, since the latter is difficult to
      // do generically.
      std::vector<typename Map::key_type> keys;
      std::vector<typename Map::mapped_type> vals;

      auto data_size = data.size();
      keys.reserve(data_size);
      vals.reserve(data_size);

      for (const auto & pr : data)
        {
          keys.push_back(pr.first);
          vals.push_back(pr.second);
        }

      this->allgather(keys, /*identical_buffer_sizes=*/false);
      this->allgather(vals, /*identical_buffer_sizes=*/false);

      data.clear();

      for (std::size_t i=0; i<keys.size(); ++i)
        {
          // Attempt to emplace this value. If it works, then the value didn't
          // already exist and we can go on. If it fails, compute the std::max
          // between the current and existing values.
          auto pr = data.emplace(keys[i], vals[i]);

          bool emplaced = pr.second;

          if (!emplaced)
            {
              auto it = pr.first;
              it->second = std::max(it->second, vals[i]);
            }
        }
    }
}

map_sum() [1/2]

template<typename Map , typename std::enable_if< std::is_base_of< DataType, StandardType< typename Map::key_type >>::value &&std::is_base_of< DataType, StandardType< typename Map::mapped_type >>::value, int >::type >

void TIMPI::Communicator::map_sum ( Map &  data ) const

inlineprivate

Private implementation function called by the map-based sum() specializations.

This is_fixed_type variant saves a communication by broadcasting pairs

Definition at line 2720 of file parallel_implementation.h.

References allgather(), and size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("sum(map)", "Parallel");

      // There may be different keys on different processors, so we
      // first gather all the (key, value) pairs and then insert
      // them, summing repeated keys, back into the map.
      //
      // Note: We don't simply use Map::value_type here because the
      // key type is const in that case and we don't have the proper
      // StandardType overloads for communicating const types.
      std::vector<std::pair<typename Map::key_type, typename Map::mapped_type>>
        vecdata(data.begin(), data.end());

      this->allgather(vecdata, /*identical_buffer_sizes=*/false);

      data.clear();
      for (const auto & pr : vecdata)
        data[pr.first] += pr.second;
    }
}

map_sum() [2/2]

template<typename Map >

void TIMPI::Communicator::map_sum ( Map &  data ) const

inlineprivate

Private implementation function called by the map-based sum() specializations.

This !is_fixed_type variant calls allgather twice: once for the keys and once for the values.

Definition at line 2752 of file parallel_implementation.h.

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("sum(map)", "Parallel");

      // There may be different keys on different processors, so we
      // first gather all the (key, value) pairs and then insert
      // them, summing repeated keys, back into the map.
      std::vector<typename Map::key_type> keys;
      std::vector<typename Map::mapped_type> vals;

      auto data_size = data.size();
      keys.reserve(data_size);
      vals.reserve(data_size);

      for (const auto & pr : data)
        {
          keys.push_back(pr.first);
          vals.push_back(pr.second);
        }

      this->allgather(keys, /*identical_buffer_sizes=*/false);
      this->allgather(vals, /*identical_buffer_sizes=*/false);

      data.clear();

      for (std::size_t i=0; i<keys.size(); ++i)
        data[keys[i]] += vals[i];
    }
}

max() [1/4]

template<typename T >

void TIMPI::Communicator::max ( const T &  r, T &  o, Request &  req  ) const

inline

Non-blocking maximum of the local value r into o with the request req.

Definition at line 2322 of file parallel_implementation.h.

References TIMPI::Request::get(), TIMPI::Request::null_request, and size().

Referenced by allgather_packed_range(), TIMPI::detail::empty_send_assertion(), gather_packed_range(), get_unique_tag(), TIMPI::detail::push_parallel_roundrobin_helper(), semiverify(), testInfinityMax(), testMapMax(), testMax(), testMaxVecBool(), testNonblockingMax(), testNonFixedTypeMapMax(), and verify().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("max()", "Parallel");

      timpi_call_mpi
        (MPI_Iallreduce (&r, &o, 1,
                         StandardType<T>(&r),
                         OpFunction<T>::max(),
                         this->get(),
                         req.get()));
    }
  else
    {
      o = r;
      req = Request::null_request;
    }
}

max() [2/4]

template<typename T >

void TIMPI::Communicator::max ( T &  r ) const

inline

Take a local variable and replace it with the maximum of it's values on all processors.

Containers are replaced element-wise.

max() [3/4]

template<typename T >

void TIMPI::Communicator::max ( T &  timpi_mpi_varr ) const

inline

Definition at line 2346 of file parallel_implementation.h.

References size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("max(scalar)", "Parallel");

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &r, 1,
                        StandardType<T>(&r),
                        OpFunction<T>::max(),
                        this->get()));
    }
}

max() [4/4]

template<typename A >

void TIMPI::Communicator::max ( std::vector< bool, A > &  r ) const

inline

Definition at line 2381 of file parallel_implementation.h.

References size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("max(vector<bool>)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<unsigned int> ruint;
      pack_vector_bool(r, ruint);
      std::vector<unsigned int> temp(ruint.size());
      timpi_call_mpi
        (MPI_Allreduce (ruint.data(), temp.data(),
                        cast_int<int>(ruint.size()),
                        StandardType<unsigned int>(), MPI_BOR,
                        this->get()));
      unpack_vector_bool(temp, r);
    }
}

maxloc() [1/3]

template<typename T >

void TIMPI::Communicator::maxloc ( T &  r, unsigned int &  max_id  ) const

inline

Take a local variable and replace it with the maximum of it's values on all processors, returning the minimum rank of a processor which originally held the maximum value.

Definition at line 2519 of file parallel_implementation.h.

References TIMPI::ignore(), TIMPI::DataPlusInt< T >::rank, rank(), size(), and TIMPI::DataPlusInt< T >::val.

Referenced by testMaxloc(), testMaxlocBool(), and testMaxlocDouble().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("maxloc(scalar)", "Parallel");

      DataPlusInt<T> data_in;
      ignore(data_in); // unused ifndef TIMPI_HAVE_MPI
      data_in.val = r;
      data_in.rank = this->rank();

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &data_in, 1,
                        dataplusint_type_acquire<T>().first,
                        OpFunction<T>::max_location(), this->get()));
      r = data_in.val;
      max_id = data_in.rank;
    }
  else
    max_id = this->rank();
}

maxloc() [2/3]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::maxloc ( std::vector< T, A1 > &  r, std::vector< unsigned int, A2 > &  max_id  ) const

inline

Take a vector of local variables and replace each entry with the maximum of it's values on all processors.

Set each min_id entry to the minimum rank where a corresponding maximum was found.

Definition at line 2544 of file parallel_implementation.h.

References rank(), size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("maxloc(vector)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<DataPlusInt<T>> data_in(r.size());
      for (std::size_t i=0; i != r.size(); ++i)
        {
          data_in[i].val  = r[i];
          data_in[i].rank = this->rank();
        }
      std::vector<DataPlusInt<T>> data_out(r.size());

      timpi_call_mpi
        (MPI_Allreduce (data_in.data(), data_out.data(),
                        cast_int<int>(r.size()),
                        dataplusint_type_acquire<T>().first,
                        OpFunction<T>::max_location(),
                        this->get()));
      for (std::size_t i=0; i != r.size(); ++i)
        {
          r[i]      = data_out[i].val;
          max_id[i] = data_out[i].rank;
        }
    }
  else if (!r.empty())
    {
      for (std::size_t i=0; i != r.size(); ++i)
        max_id[i] = this->rank();
    }
}

maxloc() [3/3]

template<typename A1 , typename A2 >

void TIMPI::Communicator::maxloc ( std::vector< bool, A1 > &  r, std::vector< unsigned int, A2 > &  max_id  ) const

inline

Definition at line 2582 of file parallel_implementation.h.

References rank(), size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("maxloc(vector<bool>)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<DataPlusInt<int>> data_in(r.size());
      for (std::size_t i=0; i != r.size(); ++i)
        {
          data_in[i].val  = r[i];
          data_in[i].rank = this->rank();
        }
      std::vector<DataPlusInt<int>> data_out(r.size());
      timpi_call_mpi
        (MPI_Allreduce (data_in.data(), data_out.data(),
                        cast_int<int>(r.size()),
                        StandardType<int>(),
                        OpFunction<int>::max_location(),
                        this->get()));
      for (std::size_t i=0; i != r.size(); ++i)
        {
          r[i]      = data_out[i].val;
          max_id[i] = data_out[i].rank;
        }
    }
  else if (!r.empty())
    {
      for (std::size_t i=0; i != r.size(); ++i)
        max_id[i] = this->rank();
    }
}

min() [1/4]

template<typename T >

void TIMPI::Communicator::min ( const T &  r, T &  o, Request &  req  ) const

inline

Non-blocking minimum of the local value r into o with the request req.

Definition at line 2142 of file parallel_implementation.h.

References TIMPI::Request::get(), TIMPI::Request::null_request, and size().

Referenced by semiverify(), testInfinityMin(), testMin(), testMinLarge(), testMinVecBool(), testNonblockingMin(), testNonblockingTest(), testNonblockingWaitany(), and verify().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("min()", "Parallel");

      timpi_call_mpi
        (MPI_Iallreduce (&r, &o, 1,
                         StandardType<T>(&r),
                         OpFunction<T>::min(),
                         this->get(),
                         req.get()));
    }
  else
    {
      o = r;
      req = Request::null_request;
    }
}

min() [2/4]

template<typename T >

void TIMPI::Communicator::min ( T &  r ) const

inline

Take a local variable and replace it with the minimum of it's values on all processors.

Containers are replaced element-wise.

min() [3/4]

template<typename T >

void TIMPI::Communicator::min ( T &  timpi_mpi_varr ) const

inline

Definition at line 2167 of file parallel_implementation.h.

References size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("min(scalar)", "Parallel");

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &r, 1,
                        StandardType<T>(&r), OpFunction<T>::min(),
                        this->get()));
    }
}

min() [4/4]

template<typename A >

void TIMPI::Communicator::min ( std::vector< bool, A > &  r ) const

inline

Definition at line 2202 of file parallel_implementation.h.

References size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("min(vector<bool>)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<unsigned int> ruint;
      pack_vector_bool(r, ruint);
      std::vector<unsigned int> temp(ruint.size());
      timpi_call_mpi
        (MPI_Allreduce (ruint.data(), temp.data(),
                        cast_int<int>(ruint.size()),
                        StandardType<unsigned int>(), MPI_BAND,
                        this->get()));
      unpack_vector_bool(temp, r);
    }
}

minloc() [1/3]

template<typename T >

void TIMPI::Communicator::minloc ( T &  r, unsigned int &  min_id  ) const

inline

Take a local variable and replace it with the minimum of it's values on all processors, returning the minimum rank of a processor which originally held the minimum value.

Definition at line 2224 of file parallel_implementation.h.

References TIMPI::ignore(), TIMPI::DataPlusInt< T >::rank, rank(), size(), and TIMPI::DataPlusInt< T >::val.

Referenced by testMinloc(), testMinlocBool(), and testMinlocDouble().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("minloc(scalar)", "Parallel");

      DataPlusInt<T> data_in;
      ignore(data_in); // unused ifndef TIMPI_HAVE_MPI
      data_in.val = r;
      data_in.rank = this->rank();

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &data_in, 1,
                        dataplusint_type_acquire<T>().first,
                        OpFunction<T>::min_location(), this->get()));
      r = data_in.val;
      min_id = data_in.rank;
    }
  else
    min_id = this->rank();
}

minloc() [2/3]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::minloc ( std::vector< T, A1 > &  r, std::vector< unsigned int, A2 > &  min_id  ) const

inline

Take a vector of local variables and replace each entry with the minimum of it's values on all processors.

Set each min_id entry to the minimum rank where a corresponding minimum was found.

Definition at line 2249 of file parallel_implementation.h.

References rank(), size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("minloc(vector)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<DataPlusInt<T>> data_in(r.size());
      for (std::size_t i=0; i != r.size(); ++i)
        {
          data_in[i].val  = r[i];
          data_in[i].rank = this->rank();
        }
      std::vector<DataPlusInt<T>> data_out(r.size());

      timpi_call_mpi
        (MPI_Allreduce (data_in.data(), data_out.data(),
                        cast_int<int>(r.size()),
                        dataplusint_type_acquire<T>().first,
                        OpFunction<T>::min_location(), this->get()));
      for (std::size_t i=0; i != r.size(); ++i)
        {
          r[i]      = data_out[i].val;
          min_id[i] = data_out[i].rank;
        }
    }
  else if (!r.empty())
    {
      for (std::size_t i=0; i != r.size(); ++i)
        min_id[i] = this->rank();
    }
}

minloc() [3/3]

template<typename A1 , typename A2 >

void TIMPI::Communicator::minloc ( std::vector< bool, A1 > &  r, std::vector< unsigned int, A2 > &  min_id  ) const

inline

Definition at line 2286 of file parallel_implementation.h.

References rank(), size(), and verify().

{
  if (this->size() > 1 && !r.empty())
    {
      TIMPI_LOG_SCOPE("minloc(vector<bool>)", "Parallel");

      timpi_assert(this->verify(r.size()));

      std::vector<DataPlusInt<int>> data_in(r.size());
      for (std::size_t i=0; i != r.size(); ++i)
        {
          data_in[i].val  = r[i];
          data_in[i].rank = this->rank();
        }
      std::vector<DataPlusInt<int>> data_out(r.size());
      timpi_call_mpi
        (MPI_Allreduce (data_in.data(), data_out.data(),
                        cast_int<int>(r.size()),
                        StandardType<int>(),
                        OpFunction<int>::min_location(), this->get()));
      for (std::size_t i=0; i != r.size(); ++i)
        {
          r[i]      = data_out[i].val;
          min_id[i] = data_out[i].rank;
        }
    }
  else if (!r.empty())
    {
      for (std::size_t i=0; i != r.size(); ++i)
        min_id[i] = this->rank();
    }
}

nonblocking_barrier()

void TIMPI::Communicator::nonblocking_barrier

(

Request &

req

)

const

Start a barrier that doesn't block.

Definition at line 238 of file communicator.C.

References TIMPI::Request::get(), and size().

Referenced by TIMPI::detail::push_parallel_nbx_helper().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("nonblocking_barrier()", "Communicator");
      timpi_call_mpi(MPI_Ibarrier (this->get(), req.get()));
    }
}

nonblocking_receive_packed_range() [1/3]

template<typename Context , typename OutputIter , typename T >

void TIMPI::Communicator::nonblocking_receive_packed_range ( const unsigned int  src_processor_id, Context *  context, OutputIter  out, const T *  output_type, Request &  req, Status &  stat, const MessageTag &  tag = any_tag  ) const

inline

Non-Blocking-receive range-of-pointers from one processor.

This is meant to receive messages from nonblocking_send_packed_range

Similar in design to the above receive_packed_range. However, this version requires a Request and a Status.

The Status must be a positively tested Status for a message of this type (i.e. a message does exist). It should most likely be generated by Communicator::packed_range_probe.

Definition at line 1255 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), receive(), and TIMPI::Status::size().

Referenced by possibly_receive_packed_range(), and TIMPI::push_parallel_packed_range().

{
  typedef typename Packing<T>::buffer_type buffer_t;

  // Receive serialized variable size objects as a sequence of
  // buffer_t.
  // Allocate a buffer on the heap so we don't have to free it until
  // after the Request::wait()
  std::vector<buffer_t> * buffer = new std::vector<buffer_t>(stat.size());
  this->receive(src_processor_id, *buffer, req, tag);

  // Make the Request::wait() handle unpacking the buffer
  req.add_post_wait_work
    (new PostWaitUnpackBuffer<std::vector<buffer_t>, Context, OutputIter, T>(*buffer, context, out));

  // Make the Request::wait() then handle deleting the buffer
  req.add_post_wait_work
    (new PostWaitDeleteBuffer<std::vector<buffer_t>>(buffer));

  // The MessageTag should stay registered for the Request lifetime
  req.add_post_wait_work
    (new PostWaitDereferenceTag(tag));
}

nonblocking_receive_packed_range() [2/3]

template<typename Context , typename OutputIter , typename T >

void TIMPI::Communicator::nonblocking_receive_packed_range ( const unsigned int  src_processor_id, Context *  context, OutputIter  out, const T *  output_type, Request &  req, Status &  stat, std::shared_ptr< std::vector< typename TIMPI::Packing< T >::buffer_type >> &  buffer, const MessageTag &  tag = any_tag  ) const

inline

Non-Blocking-receive range-of-pointers from one processor.

This is meant to receive messages from nonblocking_send_packed_range

Similar in design to the above receive_packed_range. However, this version requires a Request and a Status.

The Status must be a positively tested Status for a message of this type (i.e. a message does exist). It should most likely be generated by Communicator::packed_range_probe.

nonblocking_receive_packed_range() [3/3]

template<typename Context , typename OutputIter , typename T >

void TIMPI::Communicator::nonblocking_receive_packed_range ( const unsigned int  src_processor_id, Context *  context, OutputIter  out, const T *  , Request &  req, Status &  stat, std::shared_ptr< std::vector< typename Packing< T >::buffer_type >> &  buffer, const MessageTag &  tag  ) const

inline

Definition at line 1893 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), receive(), and TIMPI::Status::size().

{
  // If they didn't pass in a buffer - let's make one
  if (buffer == nullptr)
    buffer = std::make_shared<std::vector<typename Packing<T>::buffer_type>>();
  else
    buffer->clear();

  // Receive serialized variable size objects as a sequence of
  // buffer_t.
  // Allocate a buffer on the heap so we don't have to free it until
  // after the Request::wait()
  buffer->resize(stat.size());
  this->receive(src_processor_id, *buffer, req, tag);

  // Make the Request::wait() handle unpacking the buffer
  req.add_post_wait_work
    (new PostWaitUnpackBuffer<std::vector<typename Packing<T>::buffer_type>, Context, OutputIter, T>(*buffer, context, out));

  // Make it dereference the shared pointer (possibly freeing the buffer)
  req.add_post_wait_work
    (new PostWaitDereferenceSharedPtr<std::vector<typename Packing<T>::buffer_type>>(buffer));
}

nonblocking_send_packed_range() [1/3]

template<typename Context , typename Iter >

void TIMPI::Communicator::nonblocking_send_packed_range ( const unsigned int  dest_processor_id, const Context *  context, Iter  range_begin, const Iter  range_end, Request &  req, const MessageTag &  tag = no_tag  ) const

inline

Similar to the above Nonblocking send_packed_range with a few important differences:

1. The total size of the packed buffer MUST be less than std::numeric_limits<int>::max()
2. Only one message is generated
3. On the receiving end the message should be tested for using Communicator::packed_range_probe()
4. The message must be received by Communicator::nonblocking_receive_packed_range()

Definition at line 750 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::pack_range(), and send().

Referenced by TIMPI::push_parallel_packed_range().

{
  // Allocate a buffer on the heap so we don't have to free it until
  // after the Request::wait()
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  if (range_begin != range_end)
    {
      std::vector<buffer_t> * buffer = new std::vector<buffer_t>();

      range_begin =
        pack_range(context,
                   range_begin,
                   range_end,
                   *buffer,
                   // MPI-2 can only use integers for size
                   std::numeric_limits<int>::max());

      if (range_begin != range_end)
        timpi_error_msg("Non-blocking packed range sends cannot exceed " << std::numeric_limits<int>::max() << "in size");

      // Make the Request::wait() handle deleting the buffer
      req.add_post_wait_work
        (new PostWaitDeleteBuffer<std::vector<buffer_t>>
         (buffer));

      // Non-blocking send of the buffer
      this->send(dest_processor_id, *buffer, req, tag);
    }
}

nonblocking_send_packed_range() [2/3]

template<typename Context , typename Iter >

void TIMPI::Communicator::nonblocking_send_packed_range ( const unsigned int  dest_processor_id, const Context *  context, Iter  range_begin, const Iter  range_end, Request &  req, std::shared_ptr< std::vector< typename TIMPI::Packing< typename std::iterator_traits< Iter >::value_type >::buffer_type >> &  buffer, const MessageTag &  tag = no_tag  ) const

inline

Similar to the above Nonblocking send_packed_range with a few important differences:

1. The total size of the packed buffer MUST be less than std::numeric_limits<int>::max()
2. Only one message is generated
3. On the receiving end the message should be tested for using Communicator::packed_range_probe()
4. The message must be received by Communicator::nonblocking_receive_packed_range()

nonblocking_send_packed_range() [3/3]

template<typename Context , typename Iter >

void TIMPI::Communicator::nonblocking_send_packed_range ( const unsigned int  dest_processor_id, const Context *  context, Iter  range_begin, const Iter  range_end, Request &  req, std::shared_ptr< std::vector< typename Packing< typename std::iterator_traits< Iter >::value_type >::buffer_type >> &  buffer, const MessageTag &  tag  ) const

inline

Definition at line 1543 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::pack_range(), and send().

{
  // Allocate a buffer on the heap so we don't have to free it until
  // after the Request::wait()
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  if (range_begin != range_end)
    {
      if (buffer == nullptr)
        buffer = std::make_shared<std::vector<buffer_t>>();
      else
        buffer->clear();

      range_begin =
        pack_range(context,
                   range_begin,
                   range_end,
                   *buffer,
                   // MPI-2 can only use integers for size
                   std::numeric_limits<int>::max());

      if (range_begin != range_end)
        timpi_error_msg("Non-blocking packed range sends cannot exceed " << std::numeric_limits<int>::max() << "in size");

      // Make it dereference the shared pointer (possibly freeing the buffer)
      req.add_post_wait_work
        (new PostWaitDereferenceSharedPtr<std::vector<buffer_t>>(buffer));

      // Non-blocking send of the buffer
      this->send(dest_processor_id, *buffer, req, tag);
    }
}

operator=() [1/3]

Communicator& TIMPI::Communicator::operator= ( const Communicator &  )

delete

operator=() [2/3]

Communicator& TIMPI::Communicator::operator= ( Communicator &&  )

default

operator=() [3/3]

Communicator & TIMPI::Communicator::operator=

(

const communicator &

comm

)

Definition at line 177 of file communicator.C.

References assign(), and clear().

{
  this->clear();
  this->assign(comm);
  return *this;
}

packed_range_probe()

template<typename T >

Status TIMPI::Communicator::packed_range_probe ( const unsigned int  src_processor_id, const MessageTag &  tag, bool &  flag  ) const

inline

Non-Blocking message probe for a packed range message.

Allows information about a message to be examined before the message is actually received.

Template type must match the object type that will be in the packed range

Parameters

src_processor_id	The processor the message is expected from or TIMPI::any_source
tag	The message tag or TIMPI::any_tag
flag	Output. True if a message exists. False otherwise.

Definition at line 4040 of file parallel_implementation.h.

References TIMPI::any_source, TIMPI::Status::get(), TIMPI::ignore(), size(), and TIMPI::MessageTag::value().

Referenced by TIMPI::push_parallel_packed_range().

{
  TIMPI_LOG_SCOPE("packed_range_probe()", "Parallel");

  ignore(src_processor_id, tag); // unused in opt mode w/o MPI

  Status stat((StandardType<typename Packing<T>::buffer_type>()));

  int int_flag = 0;

  timpi_assert(src_processor_id < this->size() ||
               src_processor_id == any_source);

  timpi_call_mpi(MPI_Iprobe(int(src_processor_id),
                            tag.value(),
                            this->get(),
                            &int_flag,
                            stat.get()));

  flag = int_flag;

  return stat;
}

packed_size_of()

template<typename T , typename A1 , typename A2 >

int TIMPI::Communicator::packed_size_of ( const std::vector< std::vector< T, A1 >, A2 > &  buf, const DataType &  type  ) const

private

Definition at line 222 of file parallel_implementation.h.

References size().

{
  // figure out how many bytes we need to pack all the data
  int packedsize=0;

  // The outer buffer size
  timpi_call_mpi
    (MPI_Pack_size (1,
                    StandardType<unsigned int>(),
                    this->get(),
                    &packedsize));

  int sendsize = packedsize;

  const std::size_t n_vecs = buf.size();

  for (std::size_t i = 0; i != n_vecs; ++i)
    {
      // The size of the ith inner buffer
      timpi_call_mpi
        (MPI_Pack_size (1,
                        StandardType<unsigned int>(),
                        this->get(),
                        &packedsize));

      sendsize += packedsize;

      // The data for each inner buffer
      timpi_call_mpi
        (MPI_Pack_size (cast_int<int>(buf[i].size()), type,
                        this->get(), &packedsize));

      sendsize += packedsize;
    }

  timpi_assert (sendsize /* should at least be 1! */);
  return sendsize;
}

possibly_receive() [1/6]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< T, A > &  buf, Request &  req, const MessageTag &  tag  ) const

inline

Nonblocking-receive from one processor with user-defined type.

Checks to see if a message can be received from the src_processor_id . If so, it starts a non-blocking receive using the passed in request and returns true

Otherwise - if there is no message to receive it returns false

Note: The buf does NOT need to be properly sized before this call this will resize the buffer automatically

Parameters

src_processor_id	The pid to receive from or "any". will be set to the actual src being received from
buf	The buffer to receive into
req	The request to use
tag	The tag to use

Definition at line 4070 of file parallel_implementation.h.

Referenced by possibly_receive(), and TIMPI::push_parallel_vector_data().

{
  T * dataptr = buf.empty() ? nullptr : buf.data();

  return this->possibly_receive(src_processor_id, buf, StandardType<T>(dataptr), req, tag);
}

possibly_receive() [2/6]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type = 0>

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< T, A > &  buf, Request &  req, const MessageTag &  tag  ) const

inline

dispatches to possibly_receive_packed_range

Parameters

src_processor_id	The pid to receive from or "any". will be set to the actual src being received from
buf	The buffer to receive into
req	The request to use
tag	The tag to use

possibly_receive() [3/6]

template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type >

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< T, A > &  buf, const DataType &  type, Request &  req, const MessageTag &  tag  ) const

inline

Nonblocking-receive from one processor with user-defined type.

As above, but with manually-specified data type.

Parameters

src_processor_id	The pid to receive from or "any". will be set to the actual src being received from
buf	The buffer to receive into
type	The intrinsic datatype to receive
req	The request to use
tag	The tag to use

Definition at line 1927 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::any_source, TIMPI::Status::get(), TIMPI::Request::get(), TIMPI::Status::size(), size(), TIMPI::Status::source(), and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("possibly_receive()", "Parallel");

  Status stat(type);

  int int_flag = 0;

  timpi_assert(src_processor_id < this->size() ||
                  src_processor_id == any_source);

  timpi_call_mpi(MPI_Iprobe(int(src_processor_id),
                            tag.value(),
                            this->get(),
                            &int_flag,
                            stat.get()));

  if (int_flag)
  {
    buf.resize(stat.size());

    src_processor_id = stat.source();

    timpi_call_mpi
      (MPI_Irecv (buf.data(),
                  cast_int<int>(buf.size()),
                  type,
                  src_processor_id,
                  tag.value(),
                  this->get(),
                  req.get()));

    // The MessageTag should stay registered for the Request lifetime
    req.add_post_wait_work
      (new PostWaitDereferenceTag(tag));
  }

  return int_flag;
}

possibly_receive() [4/6]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< T, A > &  buf, const NotADataType &  type, Request &  req, const MessageTag &  tag  ) const

inline

Nonblocking-receive from one processor with user-defined type.

Dispatches to possibly_receive_packed_range

Parameters

src_processor_id	The pid to receive from or "any". will be set to the actual src being received from
buf	The buffer to receive into
type	The packable type to receive
req	The request to use
tag	The tag to use

Definition at line 1972 of file parallel_implementation.h.

References possibly_receive_packed_range().

{
  TIMPI_LOG_SCOPE("possibly_receive()", "Parallel");

  return this->possibly_receive_packed_range(src_processor_id,
                                             (void *)(nullptr),
                                             std::inserter(buf, buf.end()),
                                             (T *)(nullptr),
                                             req,
                                             tag);
}

possibly_receive() [5/6]

template<typename T , typename A1 , typename A2 >

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< std::vector< T, A1 >, A2 > &  buf, const DataType &  type, Request &  req, const MessageTag &  tag  ) const

inline

Definition at line 1991 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::any_source, TIMPI::Status::get(), receive(), TIMPI::Status::size(), size(), TIMPI::Status::source(), and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("possibly_receive()", "Parallel");

  Status stat(type);

  int int_flag = 0;

  timpi_assert(src_processor_id < this->size() ||
                  src_processor_id == any_source);

  timpi_call_mpi(MPI_Iprobe(int(src_processor_id),
                            tag.value(),
                            this->get(),
                            &int_flag,
                            stat.get()));

  if (int_flag)
  {
    src_processor_id = stat.source();

    std::vector<char> * recvbuf =
      new std::vector<char>(stat.size(StandardType<char>()));

    this->receive(src_processor_id, *recvbuf, MPI_PACKED, req, tag);

    // When we wait on the receive, we'll unpack the temporary buffer
    req.add_post_wait_work
      (new PostWaitUnpackNestedBuffer<std::vector<std::vector<T,A1>,A2>>
         (*recvbuf, buf, type, *this));

    // And then we'll free the temporary buffer
    req.add_post_wait_work
      (new PostWaitDeleteBuffer<std::vector<char>>(recvbuf));

    // The MessageTag should stay registered for the Request lifetime
    req.add_post_wait_work
      (new PostWaitDereferenceTag(tag));
  }

  return int_flag;
}

possibly_receive() [6/6]

template<typename T , typename A1 , typename A2 >

bool TIMPI::Communicator::possibly_receive ( unsigned int &  src_processor_id, std::vector< std::vector< T, A1 >, A2 > &  buf, Request &  req, const MessageTag &  tag  ) const

inline

Definition at line 4098 of file parallel_implementation.h.

References possibly_receive().

{
  T * dataptr = buf.empty() ? nullptr : (buf[0].empty() ? nullptr : buf[0].data());

  return this->possibly_receive(src_processor_id, buf, StandardType<T>(dataptr), req, tag);
}

possibly_receive_packed_range()

template<typename Context , typename OutputIter , typename T >

bool TIMPI::Communicator::possibly_receive_packed_range ( unsigned int &  src_processor_id, Context *  context, OutputIter  out, const T *  output_type, Request &  req, const MessageTag &  tag  ) const

inline

Nonblocking packed range receive from one processor with user-defined type.

Checks to see if a message can be received from the src_processor_id . If so, it starts a nonblocking packed range receive using the passed in request and returns true

Otherwise - if there is no message to receive it returns false

void Parallel::unpack(const T *, OutputIter data, const Context *) is used to unserialize type T

Parameters

src_processor_id	The pid to receive from or "any". will be set to the actual src being receieved from
context	Context pointer that will be passed into the unpack functions
out	The output iterator
output_type	The intrinsic datatype to receive
req	The request to use
tag	The tag to use

Definition at line 4110 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::any_source, nonblocking_receive_packed_range(), and size().

Referenced by possibly_receive(), and TIMPI::push_parallel_packed_range().

{
  TIMPI_LOG_SCOPE("possibly_receive_packed_range()", "Parallel");

  bool int_flag = 0;

  auto stat = packed_range_probe<T>(src_processor_id, tag, int_flag);

  if (int_flag)
  {
    src_processor_id = stat.source();

    nonblocking_receive_packed_range(src_processor_id,
                                     context,
                                     out,
                                     type,
                                     req,
                                     stat,
                                     tag);

     // The MessageTag should stay registered for the Request lifetime
     req.add_post_wait_work
       (new PostWaitDereferenceTag(tag));
  }

  timpi_assert(!int_flag || (int_flag &&
                             src_processor_id < this->size() &&
                             src_processor_id != any_source));

  return int_flag;
}

probe()

status TIMPI::Communicator::probe	(	const unsigned int	src_processor_id,
		const MessageTag &	tag = any_tag
	)		const

Blocking message probe.

Allows information about a message to be examined before the message is actually received.

Definition at line 283 of file communicator.C.

References TIMPI::any_source, TIMPI::ignore(), size(), and TIMPI::MessageTag::value().

Referenced by TIMPI::pull_parallel_vector_data(), TIMPI::detail::push_parallel_alltoall_helper(), and receive().

{
  TIMPI_LOG_SCOPE("probe()", "Communicator");

#ifndef TIMPI_HAVE_MPI
  timpi_not_implemented();
  ignore(src_processor_id, tag);
#endif

  status stat;

  timpi_assert(src_processor_id < this->size() ||
                  src_processor_id == any_source);

  timpi_call_mpi
    (MPI_Probe (int(src_processor_id), tag.value(), this->get(), &stat));

  return stat;
}

rank()

processor_id_type TIMPI::Communicator::rank ( ) const

inline

Definition at line 206 of file communicator.h.

References _rank.

Referenced by broadcast(), broadcast_packed_range(), TIMPI::detail::empty_send_assertion(), fill_data(), fill_scalar_data(), fill_vector_data(), gather(), map_broadcast(), maxloc(), minloc(), TIMPI::pull_parallel_vector_data(), TIMPI::detail::push_parallel_alltoall_helper(), TIMPI::detail::push_parallel_nbx_helper(), TIMPI::detail::push_parallel_roundrobin_helper(), TIMPI::report_here(), scatter(), send_receive(), send_receive_packed_range(), testAllGather(), testAllGatherHalfEmptyVectorString(), testAllGatherString(), testAllGatherVectorString(), testAllGatherVectorVector(), testAllGatherVectorVectorPacked(), testArrayContainerAllGather(), testBigUnion(), testBroadcast(), testBroadcastArrayType(), testBroadcastNestedType(), testBroadcastString(), testContainerAllGather(), testContainerBroadcast(), testContainerSendReceive(), testEmptyEntry(), testGather(), testGatherString(), testGatherString2(), testGettableStringAllGather(), testIrecvSend(), testIsendRecv(), testMapContainerAllGather(), testMapMap(), testMapMax(), testMapSet(), testMax(), testMaxloc(), testMaxlocBool(), testMaxlocDouble(), testMaxVecBool(), testMin(), testMinLarge(), testMinloc(), testMinlocBool(), testMinlocDouble(), testMinVecBool(), testMPIULongMin(), testNestingAllGather(), testNonblockingMax(), testNonblockingMin(), testNonblockingSum(), testNonblockingTest(), testNonblockingWaitany(), testNonFixedTypeMapMax(), testNonFixedTypeSum(), testNullAllGather(), testNullSendReceive(), testPackedSetUnion(), testPairContainerAllGather(), testPush(), testPushImpl(), testPushMove(), testPushMultimapImpl(), testPushMultimapVecVecImpl(), testPushPackedImpl(), testPushPackedImplMove(), testPushPackedNestedImpl(), testPushVecVecImpl(), testRecvIsendSets(), testRecvIsendVecVecs(), testScatter(), testSemiVerifyInf(), testSemiVerifyString(), testSemiVerifyType(), testSemiVerifyVector(), testSendRecvVecVecs(), testSplit(), testSplitByType(), testSum(), testSumOpFunction(), testTupleContainerAllGather(), testTupleStringAllGather(), testUnion(), testVectorOfContainersAllGather(), and testVectorOfContainersBroadcast().

{ return _rank; }

receive() [1/6]

template<typename T >

Status TIMPI::Communicator::receive ( const unsigned int  dest_processor_id, T &  buf, const MessageTag &  tag = any_tag  ) const

inline

Blocking-receive from one processor with data-defined type.

We do not currently support receives on one processor without MPI.

Definition at line 833 of file parallel_implementation.h.

References TIMPI::any_source, TIMPI::Status::get(), probe(), size(), and TIMPI::MessageTag::value().

Referenced by nonblocking_receive_packed_range(), possibly_receive(), TIMPI::pull_parallel_vector_data(), TIMPI::push_parallel_vector_data(), receive(), receive_packed_range(), testIrecvSend(), testIsendRecv(), testRecvIsendSets(), testRecvIsendVecVecs(), and testSendRecvVecVecs().

{
  TIMPI_LOG_SCOPE("receive()", "Parallel");

  // Get the status of the message, explicitly provide the
  // datatype so we can later query the size
  Status stat(this->probe(src_processor_id, tag), StandardType<T>(&buf));

  timpi_assert(src_processor_id < this->size() ||
                  src_processor_id == any_source);

  timpi_call_mpi
    (MPI_Recv (&buf, 1, StandardType<T>(&buf), src_processor_id,
               tag.value(), this->get(), stat.get()));

  return stat;
}

receive() [2/6]

template<typename T >

void TIMPI::Communicator::receive ( const unsigned int  dest_processor_id, T &  buf, Request &  req, const MessageTag &  tag = any_tag  ) const

inline

Nonblocking-receive from one processor with data-defined type.

Definition at line 856 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::any_source, TIMPI::Request::get(), size(), and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("receive()", "Parallel");

  timpi_assert(src_processor_id < this->size() ||
                  src_processor_id == any_source);

  timpi_call_mpi
    (MPI_Irecv (&buf, 1, StandardType<T>(&buf), src_processor_id,
                tag.value(), this->get(), req.get()));

  // The MessageTag should stay registered for the Request lifetime
  req.add_post_wait_work
    (new PostWaitDereferenceTag(tag));
}

receive() [3/6]

template<typename T >

Status TIMPI::Communicator::receive ( const unsigned int  dest_processor_id, T &  buf, const DataType &  type, const MessageTag &  tag = any_tag  ) const

inline

Blocking-receive from one processor with user-defined type.

If T is a container, container-of-containers, etc., then type should be the DataType of the underlying fixed-size entries in the container(s).

Definition at line 155 of file serial_implementation.h.

{ timpi_not_implemented(); return Status(); }

receive() [4/6]

template<typename T >

void TIMPI::Communicator::receive ( const unsigned int  dest_processor_id, T &  buf, const DataType &  type, Request &  req, const MessageTag &  tag = any_tag  ) const

inline

Nonblocking-receive from one processor with user-defined type.

If T is a container, container-of-containers, etc., then type should be the DataType of the underlying fixed-size entries in the container(s).

Definition at line 162 of file serial_implementation.h.

{ timpi_not_implemented(); }

receive() [5/6]

template<typename T >

Status TIMPI::Communicator::receive ( const unsigned int  src_processor_id, std::basic_string< T > &  buf, const MessageTag &  tag  ) const

inline

Definition at line 789 of file parallel_implementation.h.

References receive().

{
  std::vector<T> tempbuf;  // Officially C++ won't let us get a
                           // modifiable array from a string

  Status stat = this->receive(src_processor_id, tempbuf, tag);
  buf.assign(tempbuf.begin(), tempbuf.end());
  return stat;
}

receive() [6/6]

template<typename T >

void TIMPI::Communicator::receive ( const unsigned int  src_processor_id, std::basic_string< T > &  buf, Request &  req, const MessageTag &  tag  ) const

inline

Definition at line 804 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), and receive().

{
  // Officially C++ won't let us get a modifiable array from a
  // string, and we can't even put one on the stack for the
  // non-blocking case.
  std::vector<T> * tempbuf = new std::vector<T>();

  // We can clear the string, but the Request::wait() will need to
  // handle copying our temporary buffer to it
  buf.clear();

  req.add_post_wait_work
    (new PostWaitCopyBuffer<std::vector<T>,
     std::back_insert_iterator<std::basic_string<T>>>
     (tempbuf, std::back_inserter(buf)));

  // Make the Request::wait() then handle deleting the buffer
  req.add_post_wait_work
    (new PostWaitDeleteBuffer<std::vector<T>>(tempbuf));

  this->receive(src_processor_id, tempbuf, req, tag);
}

receive_packed_range()

template<typename Context , typename OutputIter , typename T >

void TIMPI::Communicator::receive_packed_range ( const unsigned int  dest_processor_id, Context *  context, OutputIter  out, const T *  output_type, const MessageTag &  tag = any_tag  ) const

inline

Blocking-receive range-of-pointers from one processor.

This function does not receive raw pointers, but rather constructs new objects whose contents match the objects pointed to by the sender.

The objects will be of type T = iterator_traits<OutputIter>::value_type.

Using std::back_inserter as the output iterator allows receive to fill any container type. Using some null_output_iterator allows the receive to be dealt with solely by TIMPI::unpack(), for objects whose unpack() is written so as to not leak memory when used in this fashion.

A future version of this method should be created to preallocate memory when receiving vectors...

void TIMPI::unpack(vector<int>::iterator in, T ** out, Context *) is used to unserialize type T, typically into a new heap-allocated object whose pointer is returned as *out.

unsigned int TIMPI::packed_size(const T *, vector<int>::const_iterator) is used to advance to the beginning of the next object's data.

Definition at line 1198 of file parallel_implementation.h.

References receive(), TIMPI::Status::source(), TIMPI::Status::tag(), and TIMPI::unpack_range().

Referenced by send_receive_packed_range().

{
  typedef typename Packing<T>::buffer_type buffer_t;

  // Receive serialized variable size objects as sequences of buffer_t
  std::size_t total_buffer_size = 0;
  Status stat = this->receive(src_processor_id, total_buffer_size, tag);

  // Use stat.source() and stat.tag() in subsequent receives - if
  // src_processor_id is or tag is "any" then we want to be sure we
  // try to receive messages all corresponding to the same send.

  std::size_t received_buffer_size = 0;
  while (received_buffer_size < total_buffer_size)
    {
      std::vector<buffer_t> buffer;
      this->receive(stat.source(), buffer, MessageTag(stat.tag()));
      received_buffer_size += buffer.size();
      unpack_range
        (buffer, context, out_iter, output_type);
    }
}

reference_unique_tag()

void TIMPI::Communicator::reference_unique_tag

(

int

tagvalue

)

const

Reference an already-acquired tag, so that we know it will be dereferenced multiple times before we can re-release it.

Definition at line 37 of file communicator.C.

References used_tag_values.

Referenced by TIMPI::MessageTag::MessageTag(), and TIMPI::MessageTag::operator=().

{
  // This had better be an already-acquired tag.
  timpi_assert(used_tag_values.count(tagvalue));

  used_tag_values[tagvalue]++;
}

scatter() [1/4]

template<typename T , typename A >

void TIMPI::Communicator::scatter ( const std::vector< T, A > &  data, T &  recv, const unsigned int  root_id = 0  ) const

inline

Take a vector of local variables and scatter the ith item to the ith processor in the communicator.

The result is saved into recv.

Definition at line 3443 of file parallel_implementation.h.

References TIMPI::ignore(), rank(), and size().

Referenced by scatter(), and testScatter().

{
  ignore(root_id); // Only needed for MPI and/or dbg/devel
  timpi_assert_less (root_id, this->size());

  // Do not allow the root_id to scatter a nullptr vector.
  // That would leave recv in an indeterminate state.
  timpi_assert (this->rank() != root_id || this->size() == data.size());

  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      recv = data[0];
      return;
    }

  TIMPI_LOG_SCOPE("scatter()", "Parallel");

  T * data_ptr = const_cast<T*>(data.empty() ? nullptr : data.data());
  ignore(data_ptr); // unused ifndef TIMPI_HAVE_MPI

  timpi_assert_less(root_id, this->size());

  timpi_call_mpi
    (MPI_Scatter (data_ptr, 1, StandardType<T>(data_ptr),
                  &recv, 1, StandardType<T>(&recv), root_id, this->get()));
}

scatter() [2/4]

template<typename T , typename A >

void TIMPI::Communicator::scatter ( const std::vector< T, A > &  data, std::vector< T, A > &  recv, const unsigned int  root_id = 0  ) const

inline

Take a vector of local variables and scatter the ith equal-sized chunk to the ith processor in the communicator.

The data size must be a multiple of the communicator size. The result is saved into recv buffer. The recv buffer does not have to be sized prior to this operation.

Definition at line 3477 of file parallel_implementation.h.

References broadcast(), TIMPI::ignore(), rank(), and size().

{
  timpi_assert_less (root_id, this->size());

  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      recv.assign(data.begin(), data.end());
      return;
    }

  TIMPI_LOG_SCOPE("scatter()", "Parallel");

  int recv_buffer_size = 0;
  if (this->rank() == root_id)
    {
      timpi_assert(data.size() % this->size() == 0);
      recv_buffer_size = cast_int<int>(data.size() / this->size());
    }

  this->broadcast(recv_buffer_size);
  recv.resize(recv_buffer_size);

  T * data_ptr = const_cast<T*>(data.empty() ? nullptr : data.data());
  T * recv_ptr = recv.empty() ? nullptr : recv.data();
  ignore(data_ptr, recv_ptr); // unused ifndef TIMPI_HAVE_MPI

  timpi_assert_less(root_id, this->size());

  timpi_call_mpi
    (MPI_Scatter (data_ptr, recv_buffer_size, StandardType<T>(data_ptr),
                  recv_ptr, recv_buffer_size, StandardType<T>(recv_ptr), root_id, this->get()));
}

scatter() [3/4]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::scatter ( const std::vector< T, A1 > &  data, const std::vector< int, A2 >  counts, std::vector< T, A1 > &  recv, const unsigned int  root_id = 0  ) const

inline

Take a vector of local variables and scatter the ith variable-sized chunk to the ith processor in the communicator.

The counts vector should contain the number of items for each processor. The result is saved into recv buffer. The recv buffer does not have to be sized prior to this operation.

Definition at line 3517 of file parallel_implementation.h.

References TIMPI::ignore(), rank(), scatter(), and size().

{
  timpi_assert_less (root_id, this->size());

  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      timpi_assert (counts.size() == this->size());
      recv.assign(data.begin(), data.begin() + counts[0]);
      return;
    }

  std::vector<int,A2> displacements(this->size(), 0);
  if (root_id == this->rank())
    {
      timpi_assert(counts.size() == this->size());

      // Create a displacements vector from the incoming counts vector
      unsigned int globalsize = 0;
      for (unsigned int i=0; i < this->size(); ++i)
        {
          displacements[i] = globalsize;
          globalsize += counts[i];
        }

      timpi_assert(data.size() == globalsize);
    }

  TIMPI_LOG_SCOPE("scatter()", "Parallel");

  // Scatter the buffer sizes to size remote buffers
  int recv_buffer_size = 0;
  this->scatter(counts, recv_buffer_size, root_id);
  recv.resize(recv_buffer_size);

  T * data_ptr = const_cast<T*>(data.empty() ? nullptr : data.data());
  int * count_ptr = const_cast<int*>(counts.empty() ? nullptr : counts.data());
  T * recv_ptr = recv.empty() ? nullptr : recv.data();
  ignore(data_ptr, count_ptr, recv_ptr); // unused ifndef TIMPI_HAVE_MPI

  timpi_assert_less(root_id, this->size());

  // Scatter the non-uniform chunks
  timpi_call_mpi
    (MPI_Scatterv (data_ptr, count_ptr, displacements.data(), StandardType<T>(data_ptr),
                   recv_ptr, recv_buffer_size, StandardType<T>(recv_ptr), root_id, this->get()));
}

scatter() [4/4]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::scatter ( const std::vector< std::vector< T, A1 >, A2 > &  data, std::vector< T, A1 > &  recv, const unsigned int  root_id = 0, const bool  identical_buffer_sizes = false  ) const

inline

Take a vector of vectors and scatter the ith inner vector to the ith processor in the communicator.

The result is saved into recv buffer. The recv buffer does not have to be sized prior to this operation.

Definition at line 3572 of file parallel_implementation.h.

References rank(), scatter(), and size().

{
  timpi_assert_less (root_id, this->size());

  if (this->size() == 1)
    {
      timpi_assert (!this->rank());
      timpi_assert (!root_id);
      timpi_assert (data.size() == this->size());
      recv.assign(data[0].begin(), data[0].end());
      return;
    }

  std::vector<T,A1> stacked_data;
  std::vector<int> counts;

  if (root_id == this->rank())
    {
      timpi_assert (data.size() == this->size());

      if (!identical_buffer_sizes)
        counts.resize(this->size());

      for (std::size_t i=0; i < data.size(); ++i)
        {
          if (!identical_buffer_sizes)
            counts[i] = cast_int<int>(data[i].size());
#ifndef NDEBUG
          else
            // Check that buffer sizes are indeed equal
            timpi_assert(!i || data[i-1].size() == data[i].size());
#endif
          std::copy(data[i].begin(), data[i].end(), std::back_inserter(stacked_data));
        }
    }

  if (identical_buffer_sizes)
    this->scatter(stacked_data, recv, root_id);
  else
    this->scatter(stacked_data, counts, recv, root_id);
}

semiverify()

template<typename T >

bool TIMPI::Communicator::semiverify ( const T *  r ) const

inline

Verify that a local pointer points to the same value on all processors where it is not nullptr.

Containers must have the same value in every entry.

Definition at line 2071 of file parallel_implementation.h.

References max(), min(), TIMPI::Attributes< T >::set_highest(), TIMPI::Attributes< T >::set_lowest(), and size().

Referenced by testSemiVerifyInf(), testSemiVerifyString(), testSemiVerifyType(), and testSemiVerifyVector().

{
  if (this->size() > 1 && Attributes<T>::has_min_max == true)
    {
      T tempmin, tempmax;
      if (r)
        tempmin = tempmax = *r;
      else
        {
          Attributes<T>::set_highest(tempmin);
          Attributes<T>::set_lowest(tempmax);
        }
      this->min(tempmin);
      this->max(tempmax);
      bool invalid = r && ((*r != tempmin) ||
                           (*r != tempmax));
      this->max(invalid);
      return !invalid;
    }

  static_assert(Attributes<T>::has_min_max,
                "Tried to semiverify an unverifiable type");

  return true;
}

send() [1/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const std::basic_string< T > &  buf, const MessageTag &  tag  ) const

inline

Definition at line 264 of file parallel_implementation.h.

References send_mode(), size(), SYNCHRONOUS, and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("send()", "Parallel");

  T * dataptr = buf.empty() ? nullptr : const_cast<T *>(buf.data());

  timpi_assert_less(dest_processor_id, this->size());

  timpi_call_mpi
    (((this->send_mode() == SYNCHRONOUS) ?
      MPI_Ssend : MPI_Send) (dataptr,
                             cast_int<int>(buf.size()),
                             StandardType<T>(dataptr),
                             dest_processor_id,
                             tag.value(),
                             this->get()));
}

send() [2/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const std::basic_string< T > &  buf, Request &  req, const MessageTag &  tag  ) const

inline

Definition at line 287 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::Request::get(), send_mode(), size(), SYNCHRONOUS, and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("send()", "Parallel");

  T * dataptr = buf.empty() ? nullptr : const_cast<T *>(buf.data());

  timpi_assert_less(dest_processor_id, this->size());

  timpi_call_mpi
    (((this->send_mode() == SYNCHRONOUS) ?
      MPI_Issend : MPI_Isend) (dataptr,
                               cast_int<int>(buf.size()),
                               StandardType<T>(dataptr),
                               dest_processor_id,
                               tag.value(),
                               this->get(),
                               req.get()));

  // The MessageTag should stay registered for the Request lifetime
  req.add_post_wait_work
    (new PostWaitDereferenceTag(tag));
}

send() [3/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const T &  buf, const MessageTag &  tag = no_tag  ) const

inline

Blocking-send to one processor with data-defined type.

We do not currently support sends on one processor without MPI.

Definition at line 316 of file parallel_implementation.h.

References send_mode(), size(), SYNCHRONOUS, and TIMPI::MessageTag::value().

Referenced by nonblocking_send_packed_range(), TIMPI::pull_parallel_vector_data(), TIMPI::push_parallel_vector_data(), send_packed_range(), send_receive(), testIrecvSend(), testIsendRecv(), testRecvIsendSets(), testRecvIsendVecVecs(), and testSendRecvVecVecs().

{
  TIMPI_LOG_SCOPE("send()", "Parallel");

  T * dataptr = const_cast<T*> (&buf);

  timpi_assert_less(dest_processor_id, this->size());

  timpi_call_mpi
    (((this->send_mode() == SYNCHRONOUS) ?
      MPI_Ssend : MPI_Send) (dataptr,
                             1,
                             StandardType<T>(dataptr),
                             dest_processor_id,
                             tag.value(),
                             this->get()));
}

send() [4/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const T &  buf, Request &  req, const MessageTag &  tag = no_tag  ) const

inline

Nonblocking-send to one processor with data-defined type.

Definition at line 339 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::Request::get(), send_mode(), size(), SYNCHRONOUS, and TIMPI::MessageTag::value().

{
  TIMPI_LOG_SCOPE("send()", "Parallel");

  T * dataptr = const_cast<T*>(&buf);

  timpi_assert_less(dest_processor_id, this->size());

  timpi_call_mpi
    (((this->send_mode() == SYNCHRONOUS) ?
      MPI_Issend : MPI_Isend) (dataptr,
                               1,
                               StandardType<T>(dataptr),
                               dest_processor_id,
                               tag.value(),
                               this->get(),
                               req.get()));

  // The MessageTag should stay registered for the Request lifetime
  req.add_post_wait_work
    (new PostWaitDereferenceTag(tag));
}

send() [5/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const T &  buf, const DataType &  type, const MessageTag &  tag = no_tag  ) const

inline

Blocking-send to one processor with user-defined type.

If T is a container, container-of-containers, etc., then type should be the DataType of the underlying fixed-size entries in the container(s).

Definition at line 78 of file serial_implementation.h.

{ timpi_not_implemented(); }

send() [6/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const T &  buf, const DataType &  type, Request &  req, const MessageTag &  tag = no_tag  ) const

inline

Nonblocking-send to one processor with user-defined type.

If T is a container, container-of-containers, etc., then type should be the DataType of the underlying fixed-size entries in the container(s).

Definition at line 85 of file serial_implementation.h.

{ timpi_not_implemented(); }

send() [7/7]

template<typename T >

void TIMPI::Communicator::send ( const unsigned int  dest_processor_id, const T &  buf, const NotADataType &  type, Request &  req, const MessageTag &  tag = no_tag  ) const

inline

Nonblocking-send to one processor with user-defined packable type.

Packing<T> must be defined for T

Definition at line 93 of file serial_implementation.h.

{ timpi_not_implemented(); }

send_mode() [1/2]

void TIMPI::Communicator::send_mode ( const SendMode  sm )

inline

Explicitly sets the SendMode type used for send operations.

Definition at line 326 of file communicator.h.

References _send_mode.

Referenced by duplicate(), TIMPI::detail::push_parallel_nbx_helper(), split(), split_by_type(), testIrecvSend(), and testIsendRecv().

{ _send_mode = sm; }

send_mode() [2/2]

SendMode TIMPI::Communicator::send_mode ( ) const

inline

Gets the user-requested SendMode.

Definition at line 331 of file communicator.h.

References _send_mode.

Referenced by duplicate(), send(), split(), and split_by_type().

{ return _send_mode; }

send_packed_range() [1/2]

template<typename Context , typename Iter >

void TIMPI::Communicator::send_packed_range ( const unsigned int  dest_processor_id, const Context *  context, Iter  range_begin, const Iter  range_end, const MessageTag &  tag = no_tag, std::size_t  approx_buffer_size = 1000000  ) const

inline

Blocking-send range-of-pointers to one processor.

This function does not send the raw pointers, but rather constructs new objects at the other end whose contents match the objects pointed to by the sender.

void TIMPI::pack(const T *, vector<int> & data, const Context *) is used to serialize type T onto the end of a data vector.

unsigned int TIMPI::packable_size(const T *, const Context *) is used to allow data vectors to reserve memory, and for additional error checking

The approximate maximum size (in entries; number of bytes will likely be 4x or 8x larger) to use in a single data vector buffer can be specified for performance or memory usage reasons; if the range cannot be packed into a single buffer of this size then multiple buffers and messages will be used.

Definition at line 625 of file parallel_implementation.h.

References TIMPI::pack_range(), TIMPI::packed_range_size(), and send().

Referenced by send_receive_packed_range().

{
  // We will serialize variable size objects from *range_begin to
  // *range_end as a sequence of plain data (e.g. ints) in this buffer
  typedef typename std::iterator_traits<Iter>::value_type T;

  std::size_t total_buffer_size =
    packed_range_size (context, range_begin, range_end);

  this->send(dest_processor_id, total_buffer_size, tag);

#ifdef DEBUG
  std::size_t used_buffer_size = 0;
#endif

  while (range_begin != range_end)
    {
      timpi_assert_greater (std::distance(range_begin, range_end), 0);

      std::vector<typename Packing<T>::buffer_type> buffer;

      const Iter next_range_begin = pack_range
        (context, range_begin, range_end, buffer, approx_buffer_size);

      timpi_assert_greater (std::distance(range_begin, next_range_begin), 0);

      range_begin = next_range_begin;

#ifdef DEBUG
      used_buffer_size += buffer.size();
#endif

      // Blocking send of the buffer
      this->send(dest_processor_id, buffer, tag);
    }

#ifdef DEBUG
  timpi_assert_equal_to(used_buffer_size, total_buffer_size);
#endif
}

send_packed_range() [2/2]

template<typename Context , typename Iter >

void TIMPI::Communicator::send_packed_range ( const unsigned int  dest_processor_id, const Context *  context, Iter  range_begin, const Iter  range_end, Request &  req, const MessageTag &  tag = no_tag, std::size_t  approx_buffer_size = 1000000  ) const

inline

Nonblocking-send range-of-pointers to one processor.

This function does not send the raw pointers, but rather constructs new objects at the other end whose contents match the objects pointed to by the sender.

void TIMPI::pack(const T *, vector<int> & data, const Context *) is used to serialize type T onto the end of a data vector.

unsigned int TIMPI::packable_size(const T *, const Context *) is used to allow data vectors to reserve memory, and for additional error checking

The approximate maximum size (in entries; number of bytes will likely be 4x or 8x larger) to use in a single data vector buffer can be specified for performance or memory usage reasons; if the range cannot be packed into a single buffer of this size then multiple buffers and messages will be used.

Definition at line 673 of file parallel_implementation.h.

References TIMPI::Request::add_post_wait_work(), TIMPI::Request::add_prior_request(), TIMPI::pack_range(), TIMPI::packed_range_size(), and send().

{
  // Allocate a buffer on the heap so we don't have to free it until
  // after the Request::wait()
  typedef typename std::iterator_traits<Iter>::value_type T;
  typedef typename Packing<T>::buffer_type buffer_t;

  std::size_t total_buffer_size =
    packed_range_size (context, range_begin, range_end);

  // That local variable will be gone soon; we need a send buffer that
  // will stick around.  I heard you like buffering so I put a buffer
  // for your buffer size so you can buffer the size of your buffer.
  std::size_t * total_buffer_size_buffer = new std::size_t;
  *total_buffer_size_buffer = total_buffer_size;

  // Delete the buffer size's buffer when we're done
  Request intermediate_req = request();
  intermediate_req.add_post_wait_work
    (new PostWaitDeleteBuffer<std::size_t>(total_buffer_size_buffer));
  this->send(dest_processor_id, *total_buffer_size_buffer, intermediate_req, tag);

  // And don't finish up the full request until we're done with its
  // dependencies
  req.add_prior_request(intermediate_req);

#ifdef DEBUG
  std::size_t used_buffer_size = 0;
#endif

  while (range_begin != range_end)
    {
      timpi_assert_greater (std::distance(range_begin, range_end), 0);

      std::vector<buffer_t> * buffer = new std::vector<buffer_t>();

      const Iter next_range_begin = pack_range
        (context, range_begin, range_end, *buffer, approx_buffer_size);

      timpi_assert_greater (std::distance(range_begin, next_range_begin), 0);

      range_begin = next_range_begin;

#ifdef DEBUG
      used_buffer_size += buffer->size();
#endif

      Request next_intermediate_req;

      Request * my_req = (range_begin == range_end) ? &req : &next_intermediate_req;

      // Make the Request::wait() handle deleting the buffer
      my_req->add_post_wait_work
        (new PostWaitDeleteBuffer<std::vector<buffer_t>>
         (buffer));

      // Non-blocking send of the buffer
      this->send(dest_processor_id, *buffer, *my_req, tag);

      if (range_begin != range_end)
        req.add_prior_request(*my_req);
    }
}

send_receive() [1/7]

template<typename T1 , typename T2 , typename std::enable_if< std::is_base_of< DataType, StandardType< T1 >>::value &&std::is_base_of< DataType, StandardType< T2 >>::value, int >::type >

void TIMPI::Communicator::send_receive ( const unsigned int   timpi_dbg_varsend_tgt, const T1 &  send_val, const unsigned int   timpi_dbg_varrecv_source, T2 &  recv_val, const MessageTag &  , const MessageTag &    ) const

inline

Send-receive data from one processor.

Definition at line 189 of file serial_implementation.h.

{
  timpi_assert_equal_to (send_tgt, 0);
  timpi_assert_equal_to (recv_source, 0);
  recv_val = send_val;
}

send_receive() [2/7]

template<typename T , typename A , typename std::enable_if< Has_buffer_type< Packing< T >>::value, int >::type >

void TIMPI::Communicator::send_receive ( const unsigned int   timpi_dbg_vardest_processor_id, const std::vector< T, A > &  send, const unsigned int   timpi_dbg_varsource_processor_id, std::vector< T, A > &  recv, const MessageTag &  , const MessageTag &    ) const

inline

Definition at line 238 of file serial_implementation.h.

References send().

{
  timpi_assert_equal_to (dest_processor_id, 0);
  timpi_assert_equal_to (source_processor_id, 0);
  recv = send;
}

send_receive() [3/7]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::send_receive ( const unsigned int   timpi_dbg_vardest_processor_id, const std::vector< std::vector< T, A1 >, A2 > &  send, const unsigned int   timpi_dbg_varsource_processor_id, std::vector< std::vector< T, A1 >, A2 > &  recv, const MessageTag &  , const MessageTag &    ) const

inline

Definition at line 297 of file serial_implementation.h.

References send().

{
  timpi_assert_equal_to (dest_processor_id, 0);
  timpi_assert_equal_to (source_processor_id, 0);
  recv = send;
}

send_receive() [4/7]

template<typename T1 , typename T2 , typename std::enable_if< std::is_base_of< DataType, StandardType< T1 >>::value &&std::is_base_of< DataType, StandardType< T2 >>::value, int >::type >

void TIMPI::Communicator::send_receive ( const unsigned int  dest_processor_id, const T1 &  send_data, const unsigned int  source_processor_id, T2 &  recv_data, const MessageTag &  send_tag = no_tag, const MessageTag &  recv_tag = any_tag  ) const

inline

Send data send to one processor while simultaneously receiving other data recv from a (potentially different) processor.

This overload is defined for fixed-size data; other overloads exist for many other categories.

Definition at line 1362 of file parallel_implementation.h.

References TIMPI::any_source, rank(), size(), and TIMPI::MessageTag::value().

Referenced by TIMPI::push_parallel_vector_data().

{
  TIMPI_LOG_SCOPE("send_receive()", "Parallel");

  if (dest_processor_id   == this->rank() &&
      source_processor_id == this->rank())
    {
      recv = sendvec;
      return;
    }

  timpi_assert_less(dest_processor_id, this->size());
  timpi_assert(source_processor_id < this->size() ||
                  source_processor_id == any_source);

  // MPI_STATUS_IGNORE is from MPI-2; using it with some versions of
  // MPICH may cause a crash:
  // https://bugzilla.mcs.anl.gov/globus/show_bug.cgi?id=1798
  timpi_call_mpi
    (MPI_Sendrecv(const_cast<T1*>(&sendvec), 1, StandardType<T1>(&sendvec),
                  dest_processor_id, send_tag.value(), &recv, 1,
                  StandardType<T2>(&recv), source_processor_id,
                  recv_tag.value(), this->get(), MPI_STATUS_IGNORE));
}

send_receive() [5/7]

template<typename T1 , typename T2 >

void TIMPI::Communicator::send_receive ( const unsigned int  dest_processor_id, const T1 &  send_data, const DataType &  type1, const unsigned int  source_processor_id, T2 &  recv_data, const DataType &  type2, const MessageTag &  send_tag = no_tag, const MessageTag &  recv_tag = any_tag  ) const

inline

Send data send to one processor while simultaneously receiving other data recv from a (potentially different) processor, using a user-specified MPI Dataype.

send_receive() [6/7]

template<typename T , typename A , typename std::enable_if< std::is_base_of< DataType, StandardType< T >>::value, int >::type >

void TIMPI::Communicator::send_receive ( const unsigned int  dest_processor_id, const std::vector< T, A > &  send_data, const unsigned int  source_processor_id, std::vector< T, A > &  recv_data, const MessageTag &  send_tag, const MessageTag &  recv_tag  ) const

inline

Definition at line 1341 of file parallel_implementation.h.

References send_receive_packed_range().

{
  this->send_receive_packed_range(dest_processor_id, (void *)(nullptr),
                                  send_data.begin(), send_data.end(),
                                  source_processor_id, (void *)(nullptr),
                                  std::back_inserter(recv_data),
                                  (const T *)(nullptr),
                                  send_tag, recv_tag);
}

send_receive() [7/7]

template<typename T , typename A1 , typename A2 >

void TIMPI::Communicator::send_receive ( const unsigned int  dest_processor_id, const std::vector< std::vector< T, A1 >, A2 > &  sendvec, const unsigned int  source_processor_id, std::vector< std::vector< T, A1 >, A2 > &  recv, const MessageTag &  send_tag, const MessageTag &  recv_tag  ) const

inline

Definition at line 1474 of file parallel_implementation.h.

{
  send_receive_vec_of_vec
    (dest_processor_id, sendvec, source_processor_id, recv,
     send_tag, recv_tag, *this);
}

send_receive_packed_range() [1/2]

template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >

void TIMPI::Communicator::send_receive_packed_range ( const unsigned int   timpi_dbg_vardest_processor_id, const Context1 *  context1, RangeIter  send_begin, const RangeIter  send_end, const unsigned int   timpi_dbg_varsource_processor_id, Context2 *  context2, OutputIter  out_iter, const T *  output_type, const MessageTag &  , const MessageTag &  , std::size_t    ) const

inline

Send-receive range-of-pointers from one processor.

If you call this without MPI you might be making a mistake, but we'll support it.

Definition at line 322 of file serial_implementation.h.

References TIMPI::pack_range(), and TIMPI::unpack_range().

{
  // This makes no sense on one processor unless we're deliberately
  // sending to ourself.
  timpi_assert_equal_to(dest_processor_id, 0);
  timpi_assert_equal_to(source_processor_id, 0);

  // On one processor, we just need to pack the range and then unpack
  // it again.
  typedef typename std::iterator_traits<RangeIter>::value_type T1;
  typedef typename Packing<T1>::buffer_type buffer_t;

  while (send_begin != send_end)
    {
      timpi_assert_greater (std::distance(send_begin, send_end), 0);

      // We will serialize variable size objects from *range_begin to
      // *range_end as a sequence of ints in this buffer
      std::vector<buffer_t> buffer;

      const RangeIter next_send_begin = pack_range
        (context1, send_begin, send_end, buffer);

      timpi_assert_greater (std::distance(send_begin, next_send_begin), 0);

      send_begin = next_send_begin;

      unpack_range
        (buffer, context2, out_iter, output_type);
    }
}

send_receive_packed_range() [2/2]

template<typename Context1 , typename RangeIter , typename Context2 , typename OutputIter , typename T >

void TIMPI::Communicator::send_receive_packed_range ( const unsigned int  dest_processor_id, const Context1 *  context1, RangeIter  send_begin, const RangeIter  send_end, const unsigned int  source_processor_id, Context2 *  context2, OutputIter  out, const T *  output_type, const MessageTag &  send_tag = no_tag, const MessageTag &  recv_tag = any_tag, std::size_t  approx_buffer_size = 1000000  ) const

inline

Send a range-of-pointers to one processor while simultaneously receiving another range from a (potentially different) processor.

This function does not send or receive raw pointers, but rather constructs new objects at each receiver whose contents match the objects pointed to by the sender.

The objects being sent will be of type T1 = iterator_traits<RangeIter>::value_type, and the objects being received will be of type T2 = iterator_traits<OutputIter>::value_type

void TIMPI::pack(const T1*, vector<int> & data, const Context1*) is used to serialize type T1 onto the end of a data vector.

Using std::back_inserter as the output iterator allows send_receive to fill any container type. Using some null_output_iterator allows the receive to be dealt with solely by TIMPI::unpack(), for objects whose unpack() is written so as to not leak memory when used in this fashion.

A future version of this method should be created to preallocate memory when receiving vectors...

void TIMPI::unpack(vector<int>::iterator in, T2** out, Context *) is used to unserialize type T2, typically into a new heap-allocated object whose pointer is returned as *out.

unsigned int TIMPI::packable_size(const T1*, const Context1*) is used to allow data vectors to reserve memory, and for additional error checking.

unsigned int TIMPI::packed_size(const T2*, vector<int>::const_iterator) is used to advance to the beginning of the next object's data.

Definition at line 1492 of file parallel_implementation.h.

References TIMPI::pack_range(), rank(), receive_packed_range(), send_packed_range(), TIMPI::unpack_range(), and TIMPI::Request::wait().

Referenced by TIMPI::push_parallel_packed_range(), send_receive(), testContainerSendReceive(), and testNullSendReceive().

{
  TIMPI_LOG_SCOPE("send_receive()", "Parallel");

  timpi_assert_equal_to
    ((dest_processor_id  == this->rank()),
     (source_processor_id == this->rank()));

  if (dest_processor_id   == this->rank() &&
      source_processor_id == this->rank())
    {
      // We need to pack and unpack, even if we don't need to
      // communicate the buffer, just in case user Packing
      // specializations have side effects

      typedef typename Packing<T>::buffer_type buffer_t;
      while (send_begin != send_end)
        {
          std::vector<buffer_t> buffer;
          send_begin = pack_range
            (context1, send_begin, send_end, buffer, approx_buffer_size);
          unpack_range
            (buffer, context2, out_iter, output_type);
        }
      return;
    }

  Request req;

  this->send_packed_range (dest_processor_id, context1, send_begin, send_end,
                           req, send_tag, approx_buffer_size);

  this->receive_packed_range (source_processor_id, context2, out_iter,
                              output_type, recv_tag);

  req.wait();
}

set_union() [1/2]

template<typename T >

void TIMPI::Communicator::set_union ( T &  data, const unsigned int  root_id  ) const

inline

Take a container (set, map, unordered_set, multimap, etc) of local variables on each processor, and collect their union over all processors, replacing the original on processor 0.

If the data is a map or unordered_map and entries exist on different processors with the same key and different values, then the value with the lowest processor id takes precedence.

Referenced by testBigUnion(), testMapMap(), testMapSet(), testPackedSetUnion(), and testUnion().

set_union() [2/2]

template<typename T >

void TIMPI::Communicator::set_union ( T &  data ) const

inline

Take a container of local variables on each processor, and replace it with their union over all processors, replacing the original on all processors.

size()

processor_id_type TIMPI::Communicator::size ( ) const

inline

Definition at line 208 of file communicator.h.

References _size.

Referenced by allgather(), alltoall(), barrier(), broadcast(), broadcast_packed_range(), gather(), map_broadcast(), map_max(), map_sum(), max(), maxloc(), min(), minloc(), nonblocking_barrier(), packed_range_probe(), packed_size_of(), possibly_receive(), possibly_receive_packed_range(), probe(), TIMPI::pull_parallel_vector_data(), TIMPI::detail::push_parallel_alltoall_helper(), TIMPI::detail::push_parallel_nbx_helper(), TIMPI::detail::push_parallel_roundrobin_helper(), receive(), scatter(), semiverify(), send(), send_receive(), sum(), testAllGatherString(), testArrayContainerAllGather(), testBigUnion(), testContainerAllGather(), testContainerSendReceive(), testEmptyEntry(), testGettableStringAllGather(), testIrecvSend(), testIsendRecv(), testMapContainerAllGather(), testMapMax(), testMax(), testMaxloc(), testMaxlocBool(), testMaxlocDouble(), testMaxVecBool(), testMinloc(), testMinlocBool(), testMinlocDouble(), testMinVecBool(), testNestingAllGather(), testNonblockingMax(), testNonblockingSum(), testNonFixedTypeMapMax(), testNonFixedTypeSum(), testNullSendReceive(), testPackedSetUnion(), testPairContainerAllGather(), testPull(), testPullImpl(), testPullOversized(), testPullPacked(), testPullVecVec(), testPullVecVecImpl(), testPullVecVecOversized(), testPush(), testPushImpl(), testPushMove(), testPushMultimap(), testPushMultimapImpl(), testPushMultimapOversized(), testPushMultimapVecVec(), testPushMultimapVecVecImpl(), testPushMultimapVecVecOversized(), testPushOversized(), testPushPacked(), testPushPackedImpl(), testPushPackedImplMove(), testPushPackedMove(), testPushPackedMoveOversized(), testPushPackedNestedImpl(), testPushPackedOversized(), testPushVecVec(), testPushVecVecImpl(), testPushVecVecOversized(), testRecvIsendSets(), testRecvIsendVecVecs(), testScatter(), testSendRecvVecVecs(), testSplit(), testSplitByType(), testSum(), testSumOpFunction(), testTupleContainerAllGather(), testTupleStringAllGather(), testUnion(), testVectorOfContainersAllGather(), testVectorOfContainersBroadcast(), and verify().

{ return _size; }

split()

void TIMPI::Communicator::split	(	int	color,
		int	key,
		Communicator &	target
	)		const

Definition at line 97 of file communicator.C.

References _I_duped_it, assign(), clear(), send_mode(), and sync_type().

Referenced by testSplit().

{
  target.clear();
  MPI_Comm newcomm;
  timpi_call_mpi
    (MPI_Comm_split(this->get(), color, key, &newcomm));

  target.assign(newcomm);
  target._I_duped_it = (color != MPI_UNDEFINED);
  target.send_mode(this->send_mode());
  target.sync_type(this->sync_type());
}

split_by_type()

void TIMPI::Communicator::split_by_type	(	int	split_type,
		int	key,
		info	i,
		Communicator &	target
	)		const

Definition at line 111 of file communicator.C.

References _I_duped_it, assign(), clear(), send_mode(), and sync_type().

Referenced by testSplitByType().

{
  target.clear();
  MPI_Comm newcomm;
  timpi_call_mpi
    (MPI_Comm_split_type(this->get(), split_type, key, i, &newcomm));

  target.assign(newcomm);
  target._I_duped_it = (split_type != MPI_UNDEFINED);
  target.send_mode(this->send_mode());
  target.sync_type(this->sync_type());
}

sum() [1/4]

template<typename T >

void TIMPI::Communicator::sum ( T &  r ) const

inline

Take a local variable and replace it with the sum of it's values on all processors.

Containers are replaced element-wise.

Referenced by testNonblockingSum(), testNonFixedTypeSum(), testSum(), and testSumOpFunction().

sum() [2/4]

template<typename T >

void TIMPI::Communicator::sum ( const T &  r, T &  o, Request &  req  ) const

inline

Non-blocking sum of the local value r into o with the request req.

Definition at line 2619 of file parallel_implementation.h.

References TIMPI::Request::get(), TIMPI::Request::null_request, and size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("sum()", "Parallel");

      timpi_call_mpi
        (MPI_Iallreduce (&r, &o, 1,
                         StandardType<T>(&r),
                         OpFunction<T>::sum(),
                         this->get(),
                         req.get()));
    }
  else
    {
      o = r;
      req = Request::null_request;
    }
}

sum() [3/4]

template<typename T >

void TIMPI::Communicator::sum ( T &  timpi_mpi_varr ) const

inline

Definition at line 2643 of file parallel_implementation.h.

References size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("sum()", "Parallel");

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &r, 1,
                        StandardType<T>(&r),
                        OpFunction<T>::sum(),
                        this->get()));
    }
}

sum() [4/4]

template<typename T >

void TIMPI::Communicator::sum ( std::complex< T > &  timpi_mpi_varr ) const

inline

Definition at line 2680 of file parallel_implementation.h.

References size().

{
  if (this->size() > 1)
    {
      TIMPI_LOG_SCOPE("sum()", "Parallel");

      timpi_call_mpi
        (MPI_Allreduce (MPI_IN_PLACE, &r, 2,
                        StandardType<T>(),
                        OpFunction<T>::sum(),
                        this->get()));
    }
}

sync_type() [1/3]

void TIMPI::Communicator::sync_type ( const SyncType  st )

inline

Explicitly sets the SyncType used for sync operations.

Definition at line 336 of file communicator.h.

References _sync_type.

Referenced by duplicate(), main(), TIMPI::pull_parallel_vector_data(), TIMPI::push_parallel_packed_range(), TIMPI::push_parallel_vector_data(), split(), split_by_type(), testPullImpl(), testPullVecVecImpl(), testPushMultimapImpl(), testPushMultimapVecVecImpl(), and testStringSyncType().

{ _sync_type = st; }

sync_type() [2/3]

void TIMPI::Communicator::sync_type

(

const std::string &

st

)

Sets the sync type used for sync operations via a string.

Useful for changing the sync type via a CLI arg or parameter.

Definition at line 455 of file communicator.C.

References ALLTOALL_COUNTS, NBX, SENDRECEIVE, and sync_type().

{
  SyncType type = NBX;
  if (st == "sendreceive")
    type = SENDRECEIVE;
  else if (st == "alltoall")
    type = ALLTOALL_COUNTS;
  else if (st != "nbx")
    timpi_error_msg("Unrecognized TIMPI sync type " << st);
  this->sync_type(type);
}

sync_type() [3/3]

SyncType TIMPI::Communicator::sync_type ( ) const

inline

Gets the user-requested SyncType.

Definition at line 348 of file communicator.h.

References _sync_type.

Referenced by duplicate(), split(), split_by_type(), and sync_type().

{ return _sync_type; }

verify()

template<typename T >

bool TIMPI::Communicator::verify ( const T &  r ) const

inline

Verify that a local variable has the same value on all processors.

Containers must have the same value in every entry.

Definition at line 2051 of file parallel_implementation.h.

References max(), min(), and size().

Referenced by allgather(), alltoall(), broadcast(), map_broadcast(), max(), maxloc(), min(), and minloc().

{
  if (this->size() > 1 && Attributes<T>::has_min_max == true)
    {
      T tempmin = r, tempmax = r;
      this->min(tempmin);
      this->max(tempmax);
      bool verified = (r == tempmin) &&
        (r == tempmax);
      this->min(verified);
      return verified;
    }

  static_assert(Attributes<T>::has_min_max,
                "Tried to verify an unverifiable type");

  return true;
}

Member Data Documentation

_communicator

communicator TIMPI::Communicator::_communicator

private

Definition at line 229 of file communicator.h.

Referenced by assign(), clear(), duplicate(), and get().

_I_duped_it

bool TIMPI::Communicator::_I_duped_it

private

Definition at line 242 of file communicator.h.

Referenced by clear(), duplicate(), split(), and split_by_type().

_max_tag

int TIMPI::Communicator::_max_tag

private

Definition at line 239 of file communicator.h.

Referenced by assign(), and get_unique_tag().

_next_tag

int TIMPI::Communicator::_next_tag

mutableprivate

Definition at line 237 of file communicator.h.

Referenced by assign(), and get_unique_tag().

_rank

processor_id_type TIMPI::Communicator::_rank

private

Definition at line 230 of file communicator.h.

Referenced by assign(), and rank().

_send_mode

SendMode TIMPI::Communicator::_send_mode

private

Definition at line 231 of file communicator.h.

Referenced by send_mode().

_size

processor_id_type TIMPI::Communicator::_size

private

Definition at line 230 of file communicator.h.

Referenced by assign(), and size().

_sync_type

SyncType TIMPI::Communicator::_sync_type

private

Definition at line 232 of file communicator.h.

Referenced by sync_type().

used_tag_values

std::map<int, unsigned int> TIMPI::Communicator::used_tag_values

mutableprivate

Definition at line 236 of file communicator.h.

Referenced by dereference_unique_tag(), get_unique_tag(), and reference_unique_tag().

---

The documentation for this class was generated from the following files:

• src/parallel/include/timpi/communicator.h
• src/parallel/include/timpi/parallel_implementation.h
• src/parallel/include/timpi/serial_implementation.h
• src/parallel/src/communicator.C