decomposition_mod Module Reference

Parallel decomposition to determine the grid points and data columns that are located on this process. More...

Functions/Subroutines
type(component_descriptor_type) function, public	decomposition_get_descriptor ()
	Provides the component descriptor for the core to register. More...
subroutine	init_callback (current_state)
	The initialisation hook. Will set up the appropriate decomposition. More...
subroutine	two_dim_decomposition (current_state)
	Decomposition into two dimensions. More...
subroutine	apply_two_dim_neighbour_information (current_state, y_procs, x_procs, coords)
	Will apply the two dimensional neighbour information. This implements the wrap around aspect if there is not an immediate neighbour in a dimensional direction. Currently assumes that the halo is on the entire neighbour process. More...
logical function	does_two_dim_work_with_domain_size (current_state, y_dims, x_dims)
	Determines whether or not the planned two dimensional decomposition will fit into the X and Y global solutions size. More...
subroutine	apply_dimension_bounds (current_state, dim, dim_size, dim_processes, dim_my_rank)
	Applys the local bounds (start, end and size) for a specific dimension. More...
subroutine	apply_z_dimension_information (current_state)
	Applys Z dimension information. As we always decompose into columns then Z is never split and as such is just the entire Z dimension regardless. More...
subroutine	one_dim_decomposition (current_state)
	One dimension decomposition. More...
subroutine	serial_decomposition (current_state)
	Serial decomposition. More...
subroutine	display_decomposition_information (current_state, decomp_name)
	Dumps out information about the decomposition at DEBUG level. More...
subroutine	apply_data_start_end_bounds (local_grid)
	Calculates and applys the start and end bounds of the local data. Held locally is the halo and local data, this signifies for each dimension where the local data (without halo) starts and ends. More...
subroutine	apply_halo_information_and_allocate_neighbours (current_state)
	Will apply halo size information and allocated the neighbour array data. More...

Detailed Description

Parallel decomposition to determine the grid points and data columns that are located on this process.

Function/Subroutine Documentation

apply_data_start_end_bounds()

subroutine decomposition_mod::apply_data_start_end_bounds ( type(local_grid_type), intent(inout)  local_grid )

private

Calculates and applys the start and end bounds of the local data. Held locally is the halo and local data, this signifies for each dimension where the local data (without halo) starts and ends.

Parameters

localGrid

The populated local grid

Definition at line 416 of file decomposition.F90.

    type(local_grid_type), intent(inout) :: local_grid
 
    integer :: i,n_dim
 
    !Total number of dimensions
    n_dim = 3
 
    do i = 1,n_dim
       local_grid%local_domain_start_index(i) = local_grid%halo_size(i) + 1
       local_grid%local_domain_end_index(i) = local_grid%halo_size(i) + local_grid%size(i)
    end do
    ! call log_master_log(LOG_DEBUG,"start_index(1)= "//&
    !      trim(conv_to_string(local_grid%local_domain_start_index(1)))//" end_index(1)="//&
    !      trim(conv_to_string(local_grid%local_domain_end_index(1)))//" start_index(2)="//&
    !      trim(conv_to_string(local_grid%local_domain_start_index(2)))//" end_index(2)="//&
    !      trim(conv_to_string(local_grid%local_domain_end_index(2)))//" start_index(3)="//&
    !      trim(conv_to_string(local_grid%local_domain_start_index(3)))//" end_index(3)="//&
    !      trim(conv_to_string(local_grid%local_domain_end_index(3))))
 

Here is the caller graph for this function:

apply_dimension_bounds()

subroutine decomposition_mod::apply_dimension_bounds ( type(model_state_type), intent(inout)  current_state, integer, intent(in)  dim, integer, intent(in)  dim_size, integer, intent(in)  dim_processes, integer, intent(in)  dim_my_rank  )

private

Applys the local bounds (start, end and size) for a specific dimension.

Parameters

current_state	The current model state_mod
dim	The dimension that we are applying
dimSize	The global size of the dimension
dimProcesses	Number of processes in the dimension
dimMyRank	My rank in the dimension

Definition at line 227 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
    integer, intent(in) :: dim, dim_size, dim_my_rank, dim_processes
 
    integer :: dimension_division, dimension_extra
 
    dimension_division = dim_size / dim_processes
    dimension_extra = dim_size - (dimension_division * dim_processes)
 
    current_state%local_grid%start(dim) = dimension_division*dim_my_rank + merge(dimension_extra, dim_my_rank, &
         dimension_extra .lt. dim_my_rank) + 1
    current_state%local_grid%end(dim) = (current_state%local_grid%start(dim)-1) + dimension_division + &
         merge(1, 0, dim_my_rank .lt. dimension_extra)
    current_state%local_grid%size(dim)=(current_state%local_grid%end(dim) - current_state%local_grid%start(dim)) + 1
    current_state%parallel%my_coords(dim) = dim_my_rank

Here is the caller graph for this function:

apply_halo_information_and_allocate_neighbours()

subroutine decomposition_mod::apply_halo_information_and_allocate_neighbours ( type(model_state_type), intent(inout), target  current_state )

private

Will apply halo size information and allocated the neighbour array data.

Parameters

localGrid

- The local grid

Definition at line 440 of file decomposition.F90.

    type(model_state_type), target, intent(inout) :: current_state
    integer :: n_dim, n_corners,total_halo_size_XY_dim
    integer :: halo_depth
 
    ! Read halo_depth value
    halo_depth = options_get_integer(current_state%options_database, "halo_depth")
 
    ! There is no halo_depth in Z direction
    current_state%local_grid%halo_size(z_index) = 0
    current_state%local_grid%halo_size(y_index) = halo_depth
    current_state%local_grid%halo_size(x_index) = halo_depth
 
    ! Left and right or Up and down
    total_halo_size_xy_dim  = current_state%local_grid%halo_size(x_index)*2
    ! Total number of dimensions
    n_dim = 3
    ! Total number of corners
    n_corners = 4
    allocate(current_state%local_grid%neighbours(n_dim,total_halo_size_xy_dim), &
         current_state%local_grid%corner_neighbours(n_corners,halo_depth))

Here is the caller graph for this function:

apply_two_dim_neighbour_information()

subroutine decomposition_mod::apply_two_dim_neighbour_information ( type(model_state_type), intent(inout)  current_state, integer, intent(in)  y_procs, integer, intent(in)  x_procs, integer, dimension(2), intent(in)  coords  )

private

Will apply the two dimensional neighbour information. This implements the wrap around aspect if there is not an immediate neighbour in a dimensional direction. Currently assumes that the halo is on the entire neighbour process.

Parameters

current_state	The current model state_mod
yProcs	Number of processes in the y dimension
xProcs	Number of processes in the x dimension
coords	Y-X coordinates of the current process

Definition at line 124 of file decomposition.F90.

 
    type(model_state_type), intent(inout) :: current_state
    integer, intent(in) :: y_procs, x_procs, coords(2)
 
    integer :: ierr, i, halo_depth
 
    current_state%parallel%wrapped_around=.false.
 
    call mpi_cart_shift(current_state%parallel%neighbour_comm, 0, 1, current_state%local_grid%neighbours(y_index,1), &
         current_state%local_grid%neighbours(y_index,3), ierr)
    if (current_state%local_grid%neighbours(y_index,1) .lt. 0) then
      call mpi_cart_rank(current_state%parallel%neighbour_comm,&
           (/y_procs-1, coords(2)/),  current_state%local_grid%neighbours(y_index,1), ierr)
      current_state%parallel%wrapped_around(y_index, 1)=.true.
    end if
    
    if (current_state%local_grid%neighbours(y_index,3) .lt. 0) then
      call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/0, coords(2)/),  current_state%local_grid%neighbours(y_index,3), ierr)
      current_state%parallel%wrapped_around(y_index, 2)=.true.
    end if    
 
    ! find out who is on the left and on the right
    call mpi_cart_shift(current_state%parallel%neighbour_comm, 1, 1, &
         current_state%local_grid%neighbours(x_index,1), &
         current_state%local_grid%neighbours(x_index,3), ierr)
    if (current_state%local_grid%neighbours(x_index,1) .lt. 0) then
      call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/coords(1), x_procs-1/),  current_state%local_grid%neighbours(x_index,1), ierr)
      current_state%parallel%wrapped_around(x_index, 1)=.true.
    end if
    if (current_state%local_grid%neighbours(x_index,3) .lt. 0) then
      call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/coords(1), 0/),  current_state%local_grid%neighbours(x_index,3), ierr)
      current_state%parallel%wrapped_around(x_index, 2)=.true.
    end if
 
    ! the second column is on the same rank than the first one
    halo_depth = options_get_integer(current_state%options_database, "halo_depth")
    
    do i = 1, halo_depth -1
       current_state%local_grid%neighbours(y_index,i+1) = &
            current_state%local_grid%neighbours(y_index,i)
       current_state%local_grid%neighbours(y_index,i+halo_depth + 1) = &
            current_state%local_grid%neighbours(y_index,i+halo_depth)
       current_state%local_grid%neighbours(x_index,i+1) = &
            current_state%local_grid%neighbours(x_index,i)
       current_state%local_grid%neighbours(x_index,i+halo_depth + 1) = &
            current_state%local_grid%neighbours(x_index,i+halo_depth)
    end do
    ! obtain the rank of the corner neighbours
    call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/merge(coords(1)-1, y_procs-1, coords(1) .ge. 1),   &
         merge(coords(2)-1, x_procs-1, coords(2) .ge. 1)/),   &
         current_state%local_grid%corner_neighbours(1,1), ierr)
    call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/merge(coords(1)-1, y_procs-1, coords(1) .ge. 1),   &
         merge(coords(2)+1, 0, coords(2) .lt. x_procs-1)/),   &
         current_state%local_grid%corner_neighbours(2,1), ierr)
    call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/merge(coords(1)+1, 0, coords(1) .lt. y_procs-1),   &
         merge(coords(2)-1, x_procs-1, coords(2) .ge. 1)/),   &
         current_state%local_grid%corner_neighbours(3,1), ierr)
    call mpi_cart_rank(current_state%parallel%neighbour_comm, &
         (/merge(coords(1)+1, 0, coords(1) .lt. y_procs-1),   &
         merge(coords(2)+1, 0, coords(2) .lt. x_procs-1)/),   &
         current_state%local_grid%corner_neighbours(4,1), ierr)
    
    !! TODO: hardcoded for halo depth two?
    current_state%local_grid%corner_neighbours(:,2)=current_state%local_grid%corner_neighbours(:,1)

Here is the caller graph for this function:

apply_z_dimension_information()

subroutine decomposition_mod::apply_z_dimension_information ( type(model_state_type), intent(inout)  current_state )

private

Applys Z dimension information. As we always decompose into columns then Z is never split and as such is just the entire Z dimension regardless.

Parameters

current_state

The current model state_mod

Definition at line 247 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
 
    current_state%local_grid%start(z_index) = 1
    current_state%local_grid%end(z_index) = current_state%global_grid%size(z_index)
    current_state%local_grid%size(z_index) = current_state%global_grid%size(z_index)
    current_state%parallel%my_coords(z_index) = 0
    current_state%parallel%dim_sizes(z_index) = 1

Here is the caller graph for this function:

decomposition_get_descriptor()

type(component_descriptor_type) function, public decomposition_mod::decomposition_get_descriptor

Provides the component descriptor for the core to register.

Returns

The descriptor describing this component

Definition at line 26 of file decomposition.F90.

    decomposition_get_descriptor%name="decomposition"
    decomposition_get_descriptor%version=0.1
    decomposition_get_descriptor%initialisation=>init_callback

Here is the call graph for this function:

display_decomposition_information()

subroutine decomposition_mod::display_decomposition_information ( type(model_state_type), intent(inout)  current_state, character(len=*), intent(in)  decomp_name  )

private

Dumps out information about the decomposition at DEBUG level.

Parameters

current_state	The current model state_mod
decompName	The name of the decomposition which has been selected

Definition at line 384 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
    character(len=*), intent(in) :: decomp_name
 
    if (log_get_logging_level() .le. log_debug) then
       call log_log(log_debug, decomp_name//" rank="//trim(conv_to_string(current_state%parallel%my_rank))//&
            ": W=("//trim(conv_to_string(current_state%local_grid%start(z_index)))//"->"&
            //trim(conv_to_string(current_state%local_grid%end(z_index)))//") V=("//trim(conv_to_string(&
            current_state%local_grid%start(y_index)))//"->"//trim(conv_to_string(current_state%local_grid%end(&
            y_index)))//") U=("//trim(conv_to_string(current_state%local_grid%start(x_index)))//&
            "->"//trim(conv_to_string(current_state%local_grid%end(x_index)))//")")
       call log_log(log_debug, "Neighbours of "//trim(conv_to_string(current_state%parallel%my_rank))//": y=(["//&
            trim(conv_to_string(current_state%local_grid%neighbours(y_index,1)))//"],["//&
            trim(conv_to_string(current_state%local_grid%neighbours(y_index,2)))//"]) x=(["//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,1)))//","//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,2)))//"],["//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,3)))//","//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,4)))//"])")
       call log_master_log(log_debug, "Halo size z="//&
            trim(conv_to_string(current_state%local_grid%halo_size(z_index)))//&
            " y="//trim(conv_to_string(current_state%local_grid%halo_size(y_index)))//" x="//&
            trim(conv_to_string(current_state%local_grid%halo_size(x_index))))
    end if
    call log_master_log(log_info, "Decomposed "//trim(conv_to_string(current_state%parallel%processes))//&
         " processes via '"//decomp_name// "' into z="//trim(conv_to_string(current_state%parallel%dim_sizes(&
         z_index)))//" y="//trim(conv_to_string(current_state%parallel%dim_sizes(y_index)))//" x="//&
         trim(conv_to_string(current_state%parallel%dim_sizes(x_index))))

Here is the caller graph for this function:

does_two_dim_work_with_domain_size()

logical function decomposition_mod::does_two_dim_work_with_domain_size ( type(model_state_type), intent(inout)  current_state, integer, intent(in)  y_dims, integer, intent(in)  x_dims  )

private

Determines whether or not the planned two dimensional decomposition will fit into the X and Y global solutions size.

Parameters

current_state	The current model state
y_dims	Parallel process dimensions in Y
x_dims	Parallel process dimensions in X

Definition at line 202 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
    integer, intent(in) :: y_dims, x_dims
 
    if (current_state%global_grid%active(y_index)) then
       if (floor(real(current_state%global_grid%size(y_index)) / y_dims) .lt. 2) then
          does_two_dim_work_with_domain_size=.false.
          return
       end if
    end if
    if (current_state%global_grid%active(x_index)) then
       if (floor(real(current_state%global_grid%size(x_index)) / x_dims) .lt. 2) then
          does_two_dim_work_with_domain_size=.false.
          return
       end if
    end if
    does_two_dim_work_with_domain_size=.true.

Here is the caller graph for this function:

init_callback()

subroutine decomposition_mod::init_callback ( type(model_state_type), intent(inout), target  current_state )

private

The initialisation hook. Will set up the appropriate decomposition.

Parameters

current_state

The current model state_mod

Definition at line 34 of file decomposition.F90.

    type(model_state_type), target, intent(inout) :: current_state
 
    character(len=STRING_LENGTH) :: method
 
    method=options_get_string(current_state%options_database, "decomposition_method")
 
    if (method .eq. "onedim") then
       current_state%parallel%decomposition_procedure => one_dim_decomposition
    else if (method .eq. "twodim") then
       current_state%parallel%decomposition_procedure => two_dim_decomposition
    else
       current_state%parallel%decomposition_procedure => serial_decomposition
       if (method .ne. "serial") call log_log(log_warn, "Decomposition method "//trim(method)//&
            " not recognised so defaulting to serial")
    end if

Here is the call graph for this function:

Here is the caller graph for this function:

one_dim_decomposition()

subroutine decomposition_mod::one_dim_decomposition ( type(model_state_type), intent(inout)  current_state )

private

One dimension decomposition.

Will decompose across the X or Y axis depending upon which is greater - it selects the greater to maximise the decomposition

Parameters

current_state

The current model state_mod

Definition at line 262 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
 
    integer :: x_size, y_size, split_size, dimension_division, dimension_extra
 
    call apply_halo_information_and_allocate_neighbours(current_state)
 
    current_state%parallel%wrapped_around=.false.
 
    x_size = merge(current_state%global_grid%size(x_index), 1, current_state%global_grid%active(x_index))
    y_size = merge(current_state%global_grid%size(y_index), 1, current_state%global_grid%active(y_index))
 
    split_size = merge(x_size, y_size, x_size .gt. y_size)
    dimension_division = split_size / current_state%parallel%processes
    dimension_extra = split_size - (dimension_division * current_state%parallel%processes)
 
    current_state%local_grid%active = current_state%global_grid%active
    current_state%local_grid%dimensions = current_state%global_grid%dimensions
    current_state%parallel%neighbour_comm = current_state%parallel%monc_communicator
    call apply_z_dimension_information(current_state)         
 
    if (x_size .gt. y_size) then
      ! Decompose in X
      current_state%local_grid%start(y_index)=1
      current_state%local_grid%end(y_index)=y_size
      current_state%local_grid%size(y_index)=y_size
      current_state%parallel%my_coords(y_index)=0
      current_state%parallel%dim_sizes(y_index)=1
      current_state%local_grid%start(x_index)=dimension_division*current_state%parallel%my_rank+merge(&
           dimension_extra, current_state%parallel%my_rank, dimension_extra .lt. current_state%parallel%my_rank) + 1
      current_state%local_grid%end(x_index)=(current_state%local_grid%start(x_index)-1) + dimension_division + merge(&
           1, 0, current_state%parallel%my_rank .lt. dimension_extra)
      current_state%local_grid%size(x_index)=(current_state%local_grid%end(x_index) - current_state%local_grid%start(x_index)) + 1
      current_state%parallel%my_coords(x_index) = current_state%parallel%my_rank
      current_state%parallel%dim_sizes(x_index) = current_state%parallel%processes
      current_state%local_grid%neighbours(y_index,:) = current_state%parallel%my_rank
      ! Currently assume same PID in single direction (TODO: relax this restraint)
      current_state%local_grid%neighbours(x_index,1:2) = merge(current_state%parallel%my_rank, current_state%parallel%processes, &
           current_state%parallel%my_rank .gt. 0) - 1
      current_state%parallel%wrapped_around(x_index, 1)=&
           current_state%local_grid%neighbours(x_index,1)==current_state%parallel%processes-1
      current_state%local_grid%neighbours(x_index,3:4) = merge(current_state%parallel%my_rank+1, 0, &
           current_state%parallel%my_rank .lt. current_state%parallel%processes-1)
      current_state%parallel%wrapped_around(x_index, 2)=current_state%local_grid%neighbours(x_index,3)==0
      current_state%local_grid%corner_neighbours(1,:)=current_state%local_grid%neighbours(x_index,1)
      current_state%local_grid%corner_neighbours(3,:)=current_state%local_grid%neighbours(x_index,1)
      current_state%local_grid%corner_neighbours(2,:)=current_state%local_grid%neighbours(x_index,3)
      current_state%local_grid%corner_neighbours(4,:)=current_state%local_grid%neighbours(x_index,3)
      current_state%parallel%wrapped_around(y_index, :)=.true.
    else 
      ! Decompose in Y
      current_state%local_grid%start(x_index)=1
      current_state%local_grid%end(x_index)=x_size
      current_state%local_grid%size(x_index)=x_size
      current_state%parallel%my_coords(x_index)=0
      current_state%parallel%dim_sizes(x_index)=1
      current_state%local_grid%start(y_index)=dimension_division*current_state%parallel%my_rank+merge(&
           dimension_extra, current_state%parallel%my_rank, dimension_extra .lt. current_state%parallel%my_rank) + 1
      current_state%local_grid%end(y_index)=(current_state%local_grid%start(y_index)-1) + dimension_division + merge(&
           1, 0, current_state%parallel%my_rank .lt. dimension_extra)
      current_state%local_grid%size(y_index)=(current_state%local_grid%end(y_index) - current_state%local_grid%start(y_index)) + 1
      current_state%parallel%my_coords(y_index)=current_state%parallel%my_rank
      current_state%parallel%dim_sizes(y_index) = current_state%parallel%processes
      current_state%local_grid%neighbours(x_index,:) = current_state%parallel%my_rank
      current_state%local_grid%neighbours(y_index,1:2) = merge(current_state%parallel%my_rank, current_state%parallel%processes, &
           current_state%parallel%my_rank .gt. 0) - 1
      current_state%parallel%wrapped_around(y_index, 1)=&
           current_state%local_grid%neighbours(y_index,1)==current_state%parallel%processes-1
      current_state%local_grid%neighbours(y_index,3:4) = merge(current_state%parallel%my_rank+1, 0, &
           current_state%parallel%my_rank .lt. current_state%parallel%processes-1)
      current_state%parallel%wrapped_around(y_index, 2)=current_state%local_grid%neighbours(y_index,3)==0
      current_state%local_grid%corner_neighbours(1:2,:)=current_state%local_grid%neighbours(y_index,1)
      current_state%local_grid%corner_neighbours(3:4,:)=current_state%local_grid%neighbours(y_index,3)
      current_state%parallel%wrapped_around(x_index, :)=.true.
    end if
 
    call apply_data_start_end_bounds(current_state%local_grid)
    call display_decomposition_information(current_state, "OneDim")

Here is the call graph for this function:

Here is the caller graph for this function:

serial_decomposition()

subroutine decomposition_mod::serial_decomposition ( type(model_state_type), intent(inout)  current_state )

private

Serial decomposition.

Simply places the entire domain onto the one process

Parameters

current_state

The current model state_mod

Definition at line 346 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
 
    call apply_halo_information_and_allocate_neighbours(current_state)
 
    current_state%local_grid%active = current_state%global_grid%active
    current_state%local_grid%dimensions = current_state%global_grid%dimensions
    current_state%parallel%neighbour_comm = current_state%parallel%monc_communicator
    call apply_z_dimension_information(current_state)
 
    current_state%local_grid%start(y_index)=1
    current_state%local_grid%start(x_index)=1
    current_state%parallel%my_coords(y_index)=0
    current_state%parallel%my_coords(x_index)=0
    current_state%parallel%dim_sizes(y_index)=0
    current_state%parallel%dim_sizes(x_index)=0
 
    current_state%local_grid%end(x_index)= &
         merge(current_state%global_grid%size(x_index), 1, current_state%global_grid%active(x_index))
    current_state%local_grid%size(x_index)=current_state%local_grid%end(x_index)
 
    current_state%local_grid%end(y_index)=&
         merge(current_state%global_grid%size(y_index), 1, current_state%global_grid%active(y_index))
    current_state%local_grid%size(y_index)=current_state%local_grid%end(y_index)
 
    current_state%local_grid%neighbours(x_index,:) = current_state%parallel%my_rank
    current_state%local_grid%neighbours(y_index,:) = current_state%parallel%my_rank
    current_state%local_grid%corner_neighbours=current_state%parallel%my_rank
 
    current_state%parallel%wrapped_around=.true.
 
    call apply_data_start_end_bounds(current_state%local_grid)
    call display_decomposition_information(current_state, "Serial")

Here is the call graph for this function:

Here is the caller graph for this function:

two_dim_decomposition()

subroutine decomposition_mod::two_dim_decomposition ( type(model_state_type), intent(inout)  current_state )

private

Decomposition into two dimensions.

Will currently balance between the x and y dimension as evenly as possible with any extra (odd numbered) processes being added to the X dimension

Parameters

current_state

The current model state

Definition at line 57 of file decomposition.F90.

    type(model_state_type), intent(inout) :: current_state
 
    integer :: ierr
    integer, dimension(2) :: coords, distributed_dims
 
    current_state%local_grid%active = current_state%global_grid%active
    current_state%local_grid%dimensions = current_state%global_grid%dimensions
 
    if (current_state%global_grid%dimensions .ne. 3) then
       call log_master_log(log_warn, "Two dimension decomposition selected with only "// &
            trim(conv_to_string(current_state%global_grid%dimensions)) //" so defaulting to one dimension instead")
       call one_dim_decomposition(current_state)
       return
    end if
 
    distributed_dims= (/0,0/)
    call mpi_dims_create(current_state%parallel%processes, 2, distributed_dims, ierr)
 
    current_state%parallel%dim_sizes(z_index)=1
    current_state%parallel%dim_sizes(y_index) = distributed_dims(1)
    current_state%parallel%dim_sizes(x_index) = distributed_dims(2)
 
    if (.not. does_two_dim_work_with_domain_size(current_state, current_state%parallel%dim_sizes(y_index), &
         current_state%parallel%dim_sizes(x_index))) then
       call log_master_log(log_warn, "Defaulting to one dimension decomposition due to solution size too small")
       call one_dim_decomposition(current_state)
       return
    end if
    
    call mpi_cart_create(current_state%parallel%monc_communicator, 2, (/distributed_dims(1), distributed_dims(2)/),&
         (/.false., .false./), .false., current_state%parallel%neighbour_comm, ierr)
    call mpi_cart_coords(current_state%parallel%neighbour_comm, current_state%parallel%my_rank, 2, coords, ierr)
 
    call apply_halo_information_and_allocate_neighbours(current_state)
    call apply_z_dimension_information(current_state)
 
    ! sets the limits for each rank in X and Y
    call apply_dimension_bounds(current_state, y_index, merge(current_state%global_grid%size(y_index), 1, &
         current_state%global_grid%active(y_index)), distributed_dims(1), coords(1))
    call apply_dimension_bounds(current_state, x_index, merge(current_state%global_grid%size(x_index), 1, &
         current_state%global_grid%active(x_index)), distributed_dims(2), coords(2))
 
    ! get all information about the neighbours and corner neighbours
    call apply_two_dim_neighbour_information(current_state, distributed_dims(1), distributed_dims(2), coords)
 
    call apply_data_start_end_bounds(current_state%local_grid)
 
    if (log_get_logging_level() .ge. log_debug) then
       call log_log(log_debug, "PID "//trim(conv_to_string(current_state%parallel%my_rank))//": y="//&
            trim(conv_to_string(current_state%parallel%my_coords(y_index)))//&
            " x="//trim(conv_to_string(current_state%parallel%my_coords(x_index)))//" ny-1="//&
            trim(conv_to_string(current_state%local_grid%neighbours(y_index,1)))//&
            " ny+1="//trim(conv_to_string(current_state%local_grid%neighbours(y_index,3)))//" nx-1="//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,1)))//" nx+1="//&
            trim(conv_to_string(current_state%local_grid%neighbours(x_index,3))))
    end if
    call display_decomposition_information(current_state, "TwoDim")

Here is the call graph for this function:

Here is the caller graph for this function: