LOGLO-FNO: Local Global Fourier Neural Operator

Modeling high-frequency information is a critical challenge in Scientific Machine Learning. For instance, fully turbulent flow simulations of Navier-Stokes equations at Reynolds numbers 3500 and above can generate high-frequency signals due to swirling fluid motions caused by eddies and vortices. Faithfully modeling such signals using neural networks depends on the accurate reconstruction of moderate to high frequencies. However, it has been well known that deep neural nets exhibit the so-called spectral bias toward learning low-frequency components. Meanwhile, Fourier Neural Operators (FNOs) have emerged as a popular class of data-driven models in recent years for solving Partial Differential Equations (PDEs) and for surrogate modeling in general. Although impressive results have been achieved on several PDE benchmark problems, FNOs often perform poorly in learning non-dominant frequencies characterized by local features. This limitation stems from the spectral bias inherent in neural networks and the explicit exclusion of high-frequency modes in FNO and its variants. Therefore, to mitigate these issues and improve FNO’s spectral learning capabilities to represent a broad range of frequency components, we propose two key architectural enhancements: (i) a parallel branch performing local spectral convolutions and (ii) a high-frequency propagation module. Moreover, we propose a novel frequency-sensitive loss term based on radially binned spectral errors. This introduction of a parallel branch for local convolutions reduces the number of trainable parameters by up to 50% while achieving the accuracy of baseline FNO that relies solely on global convolutions. Experiments on three challenging PDE problems in fluid mechanics (Kolmogorov Flow 2D & Turbulent Radiative Layer 3D) and biological pattern formation (Diffusion-Reaction 2D), and the qualitative and spectral analysis of predictions show the effectiveness of our method over the state-of-the-art neural operator baselines.

Because the patches do not overlap and together cover $D$, their indicator functions $\chi_{P_m}$ (1 on $P_m$, 0 elsewhere) form a partition of unity:

\[ \sum_{m=1}^{M} \chi_{P_m}(x) = 1 \quad \text{for every } x \in D, \qquad P_m \cap P_n = \emptyset \;\; (m \neq n). \]

Every point of the domain lies in exactly one patch, so a field splits into patch pieces $\chi_{P_m} u$ that add back up to $u = \sum_m \chi_{P_m} u$ with no gaps and no point counted twice. The local branch transforms each piece with its own FFT. Point at a patch below to see its indicator, and separate the patches to see the pieces.

$\chi_{P_m}$ of the selected patch: 1 on P₆, 0 elsewhere

$\sum_m \chi_{P_m}$: 1 at every point

A field $u$ on an $N_x \times N_y = 128 \times 128$ grid over $D$. The lines mark the patches $P_1, \dots, P_M$, numbered row by row. Pointing at a patch (or using the arrow keys) selects it; the small maps show its indicator and the sum of all indicators.

Fourier layer of FNO

Let $\mathbf{Z} \in \mathbb{R}^{\cdots \times N_c \times N_x \times N_y [\times N_z]}$ be the output of the (initial) lifting layer. Then,

\[ \mathbf{\Upsilon} := \sigma \Big[ \mathcal{K}(\mathbf{Z}) + \mathbf{W}_{f}*\mathbf{Z} + \mathbf{b}_{f} \Big] \] \[ \mathcal{L}(\mathbf{Z}) := \mathbf{W}_{c2}*\Big( \sigma \left[ \mathbf{W}_{c1} * \mathbf{\Upsilon} + \mathbf{b}_{c1} \right] \Big) + \mathbf{b}_{c2} + \left[ \mathbf{W}_{c} \odot \mathbf{Z} + \mathbf{b}_{c} \right] \]

where $\mathbf{W}_{\square}$ and $\mathbf{b}_{\square}$ are learnable parameters, and $\mathcal{K}(\cdot)$ is the global kernel integral.

Fourier layer of LOGLO-FNO

\[ \mathbf{\Upsilon}_g := ~\sigma \Big[ \mathcal{K}_g(\mathbf{Z}) + \mathbf{W}_{g}*\mathbf{Z} + \mathbf{b}_{g} \Big], \qquad \mathbf{\Upsilon}_l := \sigma \Big[ \mathcal{K}_l(\mathbf{\hat{Z}}) + \mathbf{W}_{l}*\mathbf{\hat{Z}} + \mathbf{b}_{l} \Big], \] \[ \mathcal{L}(\mathbf{Z}, \mathbf{\hat{Z}}, \mathbf{Z^{'}}) := \mathbf{W}_{gc2}*\Big( \sigma \left[ \mathbf{W}_{gc1}* \mathbf{\Upsilon}_g + \mathbf{b}_{gc1} \right] \Big) + \mathbf{b}_{gc2} + \left[ \mathbf{W}_{gc} \odot \mathbf{Z} + \mathbf{b}_{gc} \right] ~\textcolor{blue}{+} \] \[ \qquad \qquad \qquad \left[ \mathbf{W}_{lc} \odot \mathbf{\hat{Z}} + \mathbf{b}_{lc} \right] + \mathbf{W}_{lc2}*\Big( \sigma \left[ \mathbf{W}_{lc1}* \mathbf{\Upsilon}_l + \mathbf{b}_{lc1} \right] \Big) + \mathbf{b}_{lc2} ~~\textcolor{blue}{+} \] \[ \mathbf{W}_{hfc2}*\Big( \sigma \left[ \mathbf{W}_{hfc1}* \mathbf{Z^{'}} + \mathbf{b}_{hfc1} \right] \Big) + \mathbf{b}_{hfc2}, \]

Spectral Slicing: Which Fourier Modes Each Branch Keeps

Both kernel integrals, $\mathcal{K}_g$ and $\mathcal{K}_l$, are spectral convolutions: they transform their input with the FFT, multiply a chosen set of frequencies by learned weights, and transform back. $\mathcal{K}_g$ acts on the full field and keeps only a few low frequencies, while $\mathcal{K}_l$ acts on each patch and keeps all of them. Choosing that set is plain array slicing, and it only makes sense once you know where each frequency is stored. The interactive figures below show that layout in 2D and 3D.

(i) Where Frequencies Sit After an rfft

For a real input of size $N_x \times N_y$, torch.fft.rfft2 returns $N_x \times (N_y/2 + 1)$ complex coefficients. Its two axes are stored differently.

The first axis (frequencies $k_x$) is a full FFT axis. Index 0 holds the mean. Positive frequencies follow, and the negative frequencies wrap around to the end of the array, so the highest frequencies sit in the middle.
The last axis (frequencies $k_y$) is a half axis. For a real input, the coefficient at $-k_y$ is the complex conjugate of the one at $+k_y$. The rfft therefore stores only $k_y = 0, \dots, N_y/2$, and the negative half is implied.

Frequency stored at each array index for $N_x = N_y = 16$. Coloured cells are the four lowest frequencies on each side; the small number under each cell is its array index.

The consequence: on a full axis, the low frequencies are not one contiguous block. They live at both ends of the array.

(ii) 2D: Two Corners in the Global Branch

To keep the lowest $m_1$ frequencies of the full axis and the lowest $m_2$ of the half axis, a 2D spectral convolution takes one block from each end of the full axis and gives each block its own weight:

x_ft = torch.fft.rfft2(x)                        # [batch, channels, Nx, Ny//2 + 1]
out_ft[:, :,  :m1, :m2] = mul(x_ft[:, :,  :m1, :m2], weights1)   # low +kx
out_ft[:, :, -m1:, :m2] = mul(x_ft[:, :, -m1:, :m2], weights2)   # low −kx
x = torch.fft.irfft2(out_ft)                     # every other slot stays zero

Use the sliders to change the grid and the number of modes. Each kept slot is coloured by the weight that multiplies it.

Grid size N_x = N_y 16

m₁, full axis k_x (up to N_x/2) 4

m₂, half axis k_y 4

The rfft2 output of an $N_x \times N_y$ input. Rows are the full axis ($k_x$), columns the half axis ($k_y$). Grey slots are set to zero before the inverse transform. Each slot is shaded by the size $|\hat{u}(\mathbf{k})|$ of the field $u$ from the patch figure above at that frequency (for “Local branch”, of one $16 \times 16$ patch of it): most of the energy sits in the low-frequency corners.

(iii) Keeping Every Mode: the Local Branch

The two corners are separate blocks as long as $2 m_1 \le N_x$; at $m_1 = N_x/2$ they tile the full axis exactly. The local branch of LOGLO-FNO goes one step further: on each $P_s \times P_s$ patch it keeps every frequency. Nothing is truncated, so there is nothing to split into corners. The whole patch spectrum is one block, and a single weight multiplies it:

x_ft = torch.fft.rfft2(patches)      # [patches, channels, Ps, Ps//2 + 1], all modes
out_ft = mul(x_ft, weight)           # one weight over the whole spectrum
patches = torch.fft.irfft2(out_ft)

Choose “Local branch” in the figure above to see this case.

(iv) 3D: Four Corners, Not Eight

An rfft over three axes gives two full axes and one half axis. Low frequencies sit at both ends of each full axis and at the start of the half axis, so there are $2 \times 2 \times 1 = 4$ corner blocks, one per weight:

x_ft = torch.fft.rfftn(x, dim=[-3, -2, -1])      # [batch, channels, Nx, Ny, Nz//2 + 1]
out_ft[:, :,  :m1,  :m2, :m3] = mul(x_ft[:, :,  :m1,  :m2, :m3], weights1)   # +kx, +ky
out_ft[:, :, -m1:,  :m2, :m3] = mul(x_ft[:, :, -m1:,  :m2, :m3], weights2)   # −kx, +ky
out_ft[:, :,  :m1, -m2:, :m3] = mul(x_ft[:, :,  :m1, -m2:, :m3], weights3)   # +kx, −ky
out_ft[:, :, -m1:, -m2:, :m3] = mul(x_ft[:, :, -m1:, -m2:, :m3], weights4)   # −kx, −ky

If all three axes were full, there would be $2^3 = 8$ corners. Because the input is real, the half axis needs only its starting block, which halves the count. In general, $d$ spatial dimensions give $2^{d-1}$ corners.

Grid size N_x = N_y = N_z 16

m₁, full axis k_x (up to N_x/2) 4

m₂, full axis k_y (up to N_y/2) 4

m₃, half axis k_z 4

The rfftn output of an $N_x \times N_y \times N_z$ input; the outlined box is the whole stored spectrum and the axes start at the zero frequency. Drag to rotate, or focus the view and use the arrow keys; Reset brings back the starting view and its rotation. Each kept voxel is coloured by the weight that multiplies it.

Choose “Local branch”: with every mode of the patch kept, the four corners merge into a single block covering the whole volume, so the local branch uses one weight instead of four.

(v) What This Means for LOGLO-FNO

Both branches of LOGLO-FNO use spectral convolutions, at two very different settings.

The global branch works on the full field and keeps only a few low modes per axis. Its corners are disjoint and every weight is used.
The local branch works on each patch and keeps all of its modes. Its spectrum is a single block, so it uses one weight and one contraction, in 2D and 3D alike.

The channel mixing inside a spectral convolution costs $C_{\text{in}} \cdot C_{\text{out}}$ multiply-adds per kept slot, so the number of kept slots $K$ is what decides its cost. For an $N_x \times N_y = 128 \times 128$ field:

Branch	Weight blocks	Kept slots K
Global branch, 12 × 12 modes	2 corners of 12 × 12	288
Local branch, 16 × 16 patches	1 block of 16 × 9, shared by 64 patches	9,216

The local count barely depends on the patch size: $M = N_x N_y / P_s^2$ patches, each with $P_s (P_s/2 + 1)$ slots, comes to about $N_x N_y / 2$, roughly every slot of the whole field. So the local branch, not the global one, dominates the cost of the channel mixing, while its FFTs are the cheaper ones, as the complexity analysis below shows.

Rules of Thumb

On a full axis of length $N_x$, the lowest $m_1$ frequencies are [:m1] and [-m1:]. The two blocks are disjoint only if $2 m_1 \le N_x$.
On the half axis of length $N_y/2 + 1$, the lowest $m_2$ frequencies are [:m2], with $m_2 \le N_y/2 + 1$. There is no negative block.
$d$ spatial dimensions give $2^{d-1}$ corner blocks: 2 in 2D, 4 in 3D.
If you keep every mode, there are no corners: the whole spectrum is one block, multiplied by one weight.

Complexity Analysis of Local and Global Spectral Convolutions in Baseline FNO and LOGLO-FNO

(i) Global Branch - 2D Spectral Convolutions on 2D Spatial Data

Let $N_b$ be the batch size, $N_x$ and $N_y$ the resolutions of the spatial dimensions, $N_c$ be the width or the number of hidden channels, and $C_{\text{in}}$ and $C_{\text{out}}$ the numbers of input and output channels, typically set to the same value (e.g., 128). \[ \text{FLOPs}_{\text{FFT}} = 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot \log_2(N_x \cdot N_y) \] \[ \text{FLOPs}_{\text{IFFT}} = 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot \log_2(N_x \cdot N_y) \] Therefore, FFT computation on the global branch with the full 2D spatial resolution has a per channel complexity of $$\mathcal{O} \Big( N_x \cdot N_y \cdot \log_2(N_x \cdot N_y) \Big),$$ making it expensive for large spatial resolutions such as $2048\times2048$ or higher.

(ii) Local Branch - 2D Spectral Convolutions on 2D Spatial Data

Let $M = \frac{N_x \cdot N_y}{P_s^2}$ be the number of patches, $N_x$ and $N_y$ the resolutions of the spatial dimensions, $P_s \times P_s$ the patch size (e.g., $16 \times 16$), $N_c$ be the width or the number of hidden channels, and $C_{\text{in}}$ and $C_{\text{out}}$ the numbers of input and output channels, typically set to the same value (e.g., 128). \begin{align*} \text{FLOPs}_{\text{FFT}} &= 5 \cdot N_b \cdot C_{\text{in}} \cdot M \cdot P_s \cdot P_s \cdot \log_2(P_s \cdot P_s) \\ &= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot \log_2(P_s \cdot P_s) \end{align*} \begin{align*} \text{FLOPs}_{\text{IFFT}} &= 5 \cdot N_b \cdot C_{\text{out}} \cdot M \cdot P_s \cdot P_s \cdot \log_2(P_s \cdot P_s) \\ &= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot \log_2(P_s \cdot P_s) \end{align*} Thus, the FFT and IFFT computations on the local branch, which is operating on the patches has a per channel computational complexity of $$\mathcal{O} \Big( N_x \cdot N_y \cdot \log_2(P_s \cdot P_s)\Big).$$ Since, in practice, $$\mathcal{O}\Big(N_x \cdot N_y \cdot \log_2(P_s \cdot P_s)\Big) \ll \mathcal{O}\Big(N_x \cdot N_y \cdot \log_2(N_x \cdot N_y)\Big),$$ the FFT computations are significantly cheaper in the local branch. Moreover, it is highly parallelizable when computing FFTs since each patch can be processed independently, leveraging modern accelerators such as GPUs, TPUs, and other forms of processing units.

(iii) Global Branch - 3D Spectral Convolutions on 3D Spatial Data

Let $N_b$ be the batch size, $N_x$, $N_y$, and $N_z$ the resolutions of the spatial dimensions, $N_c$ be the width or the number of hidden channels, and $C_{\text{in}}$ and $C_{\text{out}}$ the numbers of input and output channels, typically set to the same value (e.g., 128). \[ \text{FLOPs}_{\text{FFT}} = 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z) \] \[ \text{FLOPs}_{\text{IFFT}} = 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z) \] Therefore, FFT computation on the global branch with the full 3D spatial resolution has a per channel complexity of $$\mathcal{O}\Big(N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z)\Big),$$ making it highly expensive for large values of $N_x$, $N_y$, and $N_z$ such as $512\times512\times512$ or further higher spatial resolutions.

(iv) Local Branch - 3D Spectral Convolutions on 3D Spatial Data

Let $M = \frac{N_x \cdot N_y \cdot N_z}{P_s^3}$ be the number of patches, $N_x$, $N_y$, and $N_z$ the resolutions of the spatial dimensions, $P_s \times P_s \times P_s$ the patch size (e.g., $16 \times 16 \times 16$, $32 \times 32 \times 32$, etc.), $N_c$ be the width or the number of hidden channels, and $C_{\text{in}}$ and $C_{\text{out}}$ the numbers of input and output channels, typically set to the same value (e.g., 128). \begin{align*} \text{FLOPs}_{\text{FFT}} &= 5 \cdot N_b \cdot C_{\text{in}} \cdot M \cdot P_s \cdot P_s \cdot P_s \cdot \log_2(P_s \cdot P_s \cdot P_s) \\ &= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(P_s \cdot P_s \cdot P_s) \end{align*} \begin{align*} \text{FLOPs}_{\text{IFFT}} &= 5 \cdot N_b \cdot C_{\text{out}} \cdot M \cdot P_s \cdot P_s \cdot P_s \cdot \log_2(P_s \cdot P_s \cdot P_s) \\ &= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(P_s \cdot P_s \cdot P_s) \end{align*} Thus, the FFT and IFFT computations on the local branch operating on 3D patches has a per channel computational complexity of $$\mathcal{O}\Big(N_x \cdot N_y \cdot N_z \cdot \log_2(P_s \cdot P_s \cdot P_s)\Big).$$ Since, in practice, $$\mathcal{O}\Big(N_x \cdot N_y \cdot N_z \cdot \log_2(P_s \cdot P_s \cdot P_s)\Big) \ll \mathcal{O}\Big(N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z)\Big),$$ the FFT computations are significantly cheaper in the local branch. Furthermore, it is highly parallelizable when computing FFTs since each patch can be processed independently, leveraging modern accelerators such as GPUs, TPUs, and other forms of PUs.

(v) In Numbers: FFT FLOPs of the Global and Local Branches

Plugging concrete sizes into the formulas above: both branches transform every grid point, so their FFT FLOPs differ only in the log factor, $\log_2(N_x \cdot N_y)$ for one FFT over the whole field against $\log_2(P_s \cdot P_s)$ for each patch (with $N_z$ and a third $P_s$ in 3D). On a grid with $N_x = N_y \,(= N_z)$, the local branch therefore needs $\log_2 N_x / \log_2 P_s$ times fewer FFT FLOPs, and the gap widens as the resolution grows while the patch size stays fixed. The inverse FFT costs the same again in both branches.

Global branch FFT

Local branch FFTs

Fewer FLOPs in the local branch

FFT FLOPs per grid point, per sample and channel

Global: one FFT of the whole field, 5 · log₂(N_x · N_y [· N_z]) Local: one FFT per patch, 5 · log₂(P_s · P_s [· P_s])

Grid size N_x = N_y (= N_z)

Patch size P_s

Batch size $N_b = 16$ and $C_{\text{in}} = 64$ channels; both branches scale with them in the same way, so the ratio does not depend on them.

FFT FLOPs from the formulas above for an $N_x \times N_y$ field (2D) or an $N_x \times N_y \times N_z$ field (3D) with $N_x = N_y = N_z$ and $P_s \times P_s \,(\times P_s)$ patches. The chart divides out what both branches share, $N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \,(\cdot N_z)$, leaving the log factor. Point at the chart to read any grid size, or click to select it.

Values as a table

🖥 Pseudocode - Radially Binned Spectral Energy of Errors

def RadialBinnedSpectralLoss(preds, target):
    # input data shape and params
    nb, nc, nx, ny, nt = target.size()
    iLow, iHigh = 4, 12
    Lx, Ly = 1.0, 1.0

    # Compute error in Fourier space
    err_phys = preds - target
    err_fft = torch.fft.fftn(err_phys, dim=[2, 3])
    err_fft_sq = torch.abs(err_fft)**2
    err_fft_sq_h = err_fft_sq[Ellipsis, :nx//2, :ny//2, :]

    # Create radial indices
    x = torch.arange(nx//2)
    y = torch.arange(ny//2)
    X, Y = torch.meshgrid(x, y, indexing="ij")
    radii = torch.sqrt(X**2 + Y**2).floor().to(torch.int) # Radial dist.
    max_radius = int(torch.max(radii))

    # flatten radii for binary mask
    radii_flat = radii.flatten() # (nx//2 * ny//2)

    # Spatially flatten Fourier space error; (nb, nc, nx//2 * ny//2, nt)
    err_fft_sq_flat = err_fft_sq_h.contiguous().reshape(nb, nc, -1, nt)

    # initialize output tensor to hold the Fourier error
    # for each radial bin at distance r from the origin
    err_F_vect_full = torch.zeros(nb, nc, max_radius + 1, nt)

    # Apply ‘index_add_‘ for all radii and accumulate the errors
    valid_r = radii_flat <= max_radius # binary mask to find valid radii

    # Sum for all valid radial indices
    err_F_vect_full.index_add_(2,
                               radii_flat[valid_r],
                               err_fft_sq_flat[:, :, valid_r]
                              )

    # Normalize & compute mean over batch; (nc, min(nx//2, ny//2), nt)
    nrm = (nx * ny) * Lx * Ly
    _err_F = torch.sqrt(torch.mean(err_F_vect_full, dim=0)) / nrm

    # Classify Fourier space error into three bands
    err_F = torch.zeros([nc, 3, nt])
    err_F[:, 0] += torch.mean(_err_F[:, :iLow], dim=1) # low freqs
    err_F[:, 1] += torch.mean(_err_F[:, iLow:iHigh], dim=1) # mid freqs
    err_F[:, 2] += torch.mean(_err_F[:, iHigh:], dim=1) # high freqs

    # mean or sum over the channels and time dimensions
    if reduction == "mean":
        freq_loss = torch.mean(err_F, dim=[0, -1])
    elif reduction == "sum":
        freq_loss = torch.sum(err_F, dim=[0, -1])

    return freq_loss

🏹 Training Objective

The 1-step training loss for $N$ trajectories, each comprising $T$ timesteps, is given by,

$$\theta^{*} = \arg\min_\theta \sum_{n=1}^{N} \sum_{t=1}^{T-1} \mathcal{C}(\mathcal{N}_{\theta}(u^t), u^{t+1}), \qquad \quad \mathcal{C} = \mathcal{C}_{\mathrm{MSE}} + \lambda \cdot \mathcal{C}_{\mathrm{freq}}, \quad 0 \leq \lambda \leq 1 $$

where $\mathcal{C}_{\mathrm{MSE}}$ is the mean squared error and $\mathcal{C}_{\mathrm{freq}}$ is the radially binned spectral loss of Algorithm 1, computed by RadialBinnedSpectralLoss in the pseudocode above.

High-Frequency Adaptive Gaussian Noise

Let $\mathbf{X}^{\mathrm{hf}} \in \mathbb{R}^{N_b \times N_x \times N_y \times (N_t \cdot N_c)}$ denote the input tensor of HF features, where $N_b$ is the batch size, $N_x$ and $N_y$ are the resolutions of the spatial dimensions, and $N_t \cdot N_c$ represents the combined temporal and channel dimensions. The high-frequency feature adaptive Gaussian noise $\mathbf{N}_{\mathrm{dynamic}}$ is then computed as follows:

1. Compute (per sample) Mean $\mu_b$ and Standard Deviation $\sigma_b$ of High-Frequency Features \[ \mu_b = \frac{1}{N_x \cdot N_y \cdot N_t \cdot N_c} \sum_{i=1}^{N_x} \sum_{j=1}^{N_y} \sum_{k=1}^{N_t N_c} \mathbf{X}_{b,i,j,k}^{\mathrm{hf}} \] \[ \sigma_b = \sqrt{\frac{1}{N_x \cdot N_y \cdot N_t \cdot N_c} \sum_{i=1}^{N_x} \sum_{j=1}^{N_y} \sum_{k=1}^{N_t N_c} (\mathbf{X}_{b,i,j,k}^{\mathrm{hf}} - \mu)^2} + \epsilon \] where $\epsilon$ is a small constant added for numerical stability, and $\mu$ and $\sigma$ are obtained by stacking the per sample statistics along the batch dimension.

2. Generate Standard Gaussian Noise \[ \mathbf{N} \sim \mathcal{N}(0, 1) \] 3. Scale Noise Dynamically \[ \mathbf{N}_{\mathrm{dynamic}} = \mu + \alpha \cdot \sigma \cdot \mathbf{N} \] where $\alpha$ is a small value such as 0.025 and $\mathbf{N}_{\mathrm{dynamic}}$ has the same shape as the input $\mathbf{X}^{\mathrm{hf}}$.

$\mathbf{N}_{\mathrm{dynamic}}$ can now be added to the batch of inputs to the global and local branches during the training phase of LOGLO-FNO.