MONC
Data Types | Modules | Functions/Subroutines | Variables
pencilfft.F90 File Reference

Go to the source code of this file.

Data Types

type  pencil_fft_mod::pencil_transposition
 Describes a specific pencil transposition, from one pencil decomposition to another. More...
 

Modules

module  pencil_fft_mod
 This Pencil FFT performs 3D forward and backwards FFTs using pencil decomposition. It uses FFTW for the actual FFT kernel and this module contains all the data decomposition around this. There is no FFT required in Z, so this performs FFTs in Y and X (in that order forward and reversed backwards.) The data decomposition is the complex aspect, there is the concept of forward and backwards transformations. Forward transformations will go from pencil Z to Y to X and the backwards transformations undo these, so go from X to Y to Z. Note that we use quite a lot of buffer space here, this could be cut down if Y=X dimensions so some optimisation on memory could be done there in that case.
 

Functions/Subroutines

integer function, dimension(3), public pencil_fft_mod::initialise_pencil_fft (current_state, my_y_start, my_x_start)
 Initialises the pencil FFT functionality, this will create the transposition structures needed. More...
 
subroutine, public pencil_fft_mod::finalise_pencil_fft (monc_communicator)
 Cleans up allocated buffer memory. More...
 
subroutine, public pencil_fft_mod::perform_forward_3dfft (current_state, source_data, target_data)
 Performs a forward 3D FFT and currently results in target data which is the X, Z, Y oriented pencil Note that the source_data here takes no account for the halo, it is up to caller to exclude this. This does no FFT in Z, but transposes to Y, does FFT in Y, then transposes to X and performs an FFT in that dimension. Pencil decomposition is used which has already been set up. More...
 
subroutine, public pencil_fft_mod::perform_backwards_3dfft (current_state, source_data, target_data)
 Performs a backwards 3D FFT and currently results in target data which is the X, Z, Y oriented pencil Note that the source_data here takes no account for the halo, it is up to caller to exclude this. This does no FFT in Z, but transposes to Y, does FFT in Y, then transposes to X and performs an FFT in that dimension. Pencil decomposition is used which has already been set up. More...
 
subroutine pencil_fft_mod::initialise_buffers ()
 Initialises memory for the buffers used in the FFT. More...
 
subroutine pencil_fft_mod::initialise_transpositions (current_state, y_distinct_sizes, x_distinct_sizes)
 Initialises the pencil transpositions, from a pencil in one dimension to that in another. More...
 
type(pencil_transposition) function pencil_fft_mod::create_transposition (global_grid, existing_transposition, new_pencil_dim, process_dim_sizes, direction, extended_dimensions)
 Creates a specific pencil transposition description. It is maybe more a decomposition description, but the main complexity comes from the transposition from existing decomposition to new decomposition so therefore it is called transposition. The new pencil decomposition depends not only on the dimension to split on, but also the existing pencil decomposition. The new decomposed dimension (i.e. the existing pencil dimension) * other local dimensions is used as the sending size, receiving though requires knowledge about the data size on the source process so others will send their pencil dimension size to this process. More...
 
subroutine pencil_fft_mod::transpose_and_forward_fft_in_y (current_state, source_data, buffer, real_buffer)
 Performs the transposition and forward FFT in the y dimension then converts back to real numbers. The Y size is (n/2+1)*2 due to the complex to real transformation after the FFT. More...
 
subroutine pencil_fft_mod::transpose_and_backward_fft_in_x (current_state, source_data, buffer, real_buffer)
 Performs the backwards FFT in X and then transposes to Y pencil. The FFT requires complex numbers which are converted to real, so the this real to complex operation is performed first. If n is the logical size of the FFT row, then the input size is n+2, complex number size is n/2+1 and we get n reals out. More...
 
subroutine pencil_fft_mod::transpose_and_forward_fft_in_x (current_state, buffer1, buffer2, buffer3)
 Performs the transposition and forward FFT in the x dimension. After the FFT the complex space is converted back into real numbers. The X size is (n/2+1)*2 due to this transformation. More...
 
subroutine pencil_fft_mod::transpose_and_backward_fft_in_y (current_state, source_data, buffer, real_buffer)
 Performs the backwards FFT in Y and then transposes to Z pencil. The FFT requires complex numbers which are converted to real, so the this real to complex operation is performed first. If n is the logical size of the FFT row, then the input size is n+2, complex number size is n/2+1 and we get n reals out. More...
 
subroutine pencil_fft_mod::transpose_to_pencil (transposition_description, source_dims, communicator, direction, source_data, target_data)
 Transposes globally to a new pencil decomposition. This goes from the source dimensions a,b,c to b,c,a (forwards) or c,a,b (backwards.) It requires multiple steps, first the local data is transposed to c,b,a regardless of direction. then it is communicated via alltoall, each process then assembles its own b,c,a or c,a,b data via contiguising across blocks as the data layout is nonlinear. More...
 
subroutine pencil_fft_mod::contiguise_data (transposition_description, source_dims, direction, source_real_buffer, target_real_buffer)
 Contiguises from c,b,a to b,c,a (forwards) or c,a,b (backwards) where these are defined by the source_dims argument. It is not as simple as just swapping the required dimensions, as this is after the mpi alltoall and each block lies after the previous block running sequentially in a. More...
 
subroutine pencil_fft_mod::perform_r2c_fft (source_data, transformed_data, row_size, num_rows, plan_id)
 Actually performs a forward real to complex FFT. More...
 
subroutine pencil_fft_mod::perform_c2r_fft (source_data, transformed_data, row_size, num_rows, plan_id)
 Performs the complex to real (backwards) FFT. More...
 
subroutine pencil_fft_mod::rearrange_data_for_sending (real_source, real_target)
 Rearranges data for sending, transposing a,b,c into c,b,a . This is done as alltoall splits on dimension c so to go from one pencil to another we assume here that a is the existing pencil as it is contiguous. More...
 
subroutine pencil_fft_mod::determine_my_process_sizes_per_dim (existing_pencil_dim, existing_pencil_size, new_pencil_procs_per_dim, global_grid, extended_dimensions, specific_sizes_per_dim)
 Determines the number of elements to on my process per dimension which either need to be sent to (forwards transformation) or received from (backwards) each target process (in the row or column) This depends on the existing pencil decomposition, as effectively we are breaking that contigulity and decomposing it into n blocks in that dimension now (provided by new_pencil_procs_per_dim) More...
 
subroutine pencil_fft_mod::determine_offsets_from_size (source_sizes, determined_offsets)
 Simple helper function to deduce send or receive offsets from the sizes. More...
 
integer function, dimension(3) pencil_fft_mod::determine_pencil_process_dimensions (new_pencil_dim, existing_pencil_dim, existing_pencil_procs)
 Determines the number of processes in each dimension for the target decomposition. This depends heavily on the existing decomposition, as we basically contiguise our pencil dimension and decompose the existing pencil dimension. The third dimension remains unchanged. More...
 
integer function, dimension(3) pencil_fft_mod::determine_my_pencil_location (new_pencil_dim, existing_pencil_dim, existing_locations)
 Determines my location for each dimension in the new pencil decomposition. I.e. which block I am operating on. More...
 
subroutine pencil_fft_mod::concatenate_dimension_sizes (dims, concatenated_dim_sizes)
 Concatenates sizes in multiple dimensions for each target process (in a row or column) into a product of that. This represents all the dimension sizes per process. More...
 
subroutine pencil_fft_mod::determine_matching_process_dimensions (new_pencil_dim, existing_pencil_dim, proc_sizes, my_pencil_size, pencil_processes_per_dim, specific_sizes_per_dim)
 Determines the sizes per dimension on the matching process either to receive from (forward transposition) or send to (backwards transposition) each source process. Not only does this depend on the my pencil sizes, but it also depends on the amount of data that the source process has to send over. More...
 
type(pencil_transposition) function pencil_fft_mod::create_initial_transposition_description (current_state)
 Creates an initial transposition representation of the Z pencil that MONC is normally decomposed in. This is then fed into the create transposition procedure which will generate transpositions to other pencils. More...
 
integer function, dimension(3) pencil_fft_mod::determine_pencil_size (new_pencil_dim, pencil_process_layout, my_pencil_location, existing_transposition, global_grid, extended_dimensions)
 Deduces the size of my (local) pencil based upon the new decomposition. This depends heavily on the current pencil decomposition, the new pencil dimension is the global size, the existing pencil dimension becomes decomposed based on the number of processes in that dimension. The third dimension remains unchanged. More...
 
logical function pencil_fft_mod::is_extended_dimension (dimension, extended_dimensions)
 Determines whether or not the specific dimension is in the list of extended dimensions. More...
 
integer function, dimension(size(process_dim_sizes)) pencil_fft_mod::normal_to_extended_process_dim_sizes (process_dim_sizes)
 Transforms real process dimension sizes into their real after FFT complex->real transformation. The way this works is that it goes from n to (n/2+1)*2 numbers which is distributed amongst the processes deterministically. More...
 
subroutine pencil_fft_mod::convert_complex_to_real (complex_data, real_data)
 Converts complex representation to its real data counterpart and is called after each forward FFT. After a r2c FFT, there are n/2+1 complex numbers - which means that there will be more real numbers in Fourier space than are provided into the forward FFT call (due to the extra +1). Note that the real size n will always be complex size * 2 This always unpacks the complex dimension in the first dimension. More...
 
subroutine pencil_fft_mod::convert_real_to_complex (real_data, complex_data)
 Converts reals into their complex representation, this is called for backwards FFTs as we need to feed in complex numbers to force FFTW to do a backwards. It is a relatively simple transformation, as n goes into n/2 complex numbers and as this is the result of the convert_complex_to_real procedure, n always divides evenly. This is always applied to the first dimension of the real data. More...
 
integer function pencil_fft_mod::deduce_my_global_start (current_state, dimension)
 Determines my global start coordinate in Fourier space. This is required for cos y and cos x calculation which is fed into the tridiagonal solver. After the forward FFTs, each process has ((n/2+1)/p+r) * 2 elements, where p is the number of processes and r is the uneven process remainder (1 or 0 depending on p). Therefore some processes will have t elements, and some t-2 elements to feed into the solver. More...
 

Variables

integer, parameter pencil_fft_mod::forward =1
 
integer, parameter pencil_fft_mod::backward =2
 Transposition directions. More...
 
integer pencil_fft_mod::dim_y_comm
 
integer pencil_fft_mod::dim_x_comm
 Communicators for each dimension. More...
 
type(pencil_transposition) pencil_fft_mod::y_from_z_transposition
 
type(pencil_transposition) pencil_fft_mod::x_from_y_transposition
 
type(pencil_transposition) pencil_fft_mod::y_from_x_transposition
 
type(pencil_transposition) pencil_fft_mod::z_from_y_transposition
 
type(pencil_transposition) pencil_fft_mod::y_from_z_2_transposition
 
type(pencil_transposition) pencil_fft_mod::x_from_y_2_transposition
 
type(pencil_transposition) pencil_fft_mod::y_from_x_2_transposition
 
type(pencil_transposition) pencil_fft_mod::z_from_y_2_transposition
 
real(kind=default_precision), dimension(:,:,:), pointer, contiguous pencil_fft_mod::real_buffer1
 
real(kind=default_precision), dimension(:,:,:), pointer, contiguous pencil_fft_mod::real_buffer2
 
real(kind=default_precision), dimension(:,:,:), pointer, contiguous pencil_fft_mod::real_buffer3
 
real(kind=default_precision), dimension(:,:,:), pointer, contiguous pencil_fft_mod::fft_in_y_buffer
 
real(kind=default_precision), dimension(:,:,:), pointer, contiguous pencil_fft_mod::fft_in_x_buffer
 
complex(c_double_complex), dimension(:,:,:), pointer, contiguous pencil_fft_mod::buffer1
 
complex(c_double_complex), dimension(:,:,:), pointer, contiguous pencil_fft_mod::buffer2
 
type(c_ptr), dimension(4) pencil_fft_mod::fftw_plan
 
logical, dimension(4) pencil_fft_mod::fftw_plan_initialised =.false.