List of Predefined Architectures#

Deep-Set-Based NPs#

neuralprocesses.architectures.gnp.construct_gnp(dim_x=1, dim_y=1, dim_yc=None, dim_yt=None, dim_embedding=256, attention=False, attention_num_heads=8, num_enc_layers=3, enc_same=False, num_dec_layers=6, width=512, nonlinearity='ReLU', likelihood='lowrank', num_basis_functions=512, dim_lv=0, lv_likelihood='het', transform=None, dtype=None, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>)[source]#

A Gaussian Neural Process.

Parameters:

  • dim_x (int, optional) – Dimensionality of the inputs. Defaults to 1.

  • dim_y (int, optional) – Dimensionality of the outputs. Defaults to 1.

  • dim_yc (int or tuple[int], optional) – Dimensionality of the outputs of the context set. You should set this if the dimensionality of the outputs of the context set is not equal to the dimensionality of the outputs of the target set. You should also set this if you want to use multiple context sets. In that case, set this equal to a tuple of integers indicating the respective output dimensionalities.

  • dim_yt (int, optional) – Dimensionality of the outputs of the target set. You should set this if the dimensionality of the outputs of the target set is not equal to the dimensionality of the outputs of the context set.

  • dim_embedding (int, optional) – Dimensionality of the embedding. Defaults to 128.

  • attention (bool, optional) – Use attention for the deterministic encoder. Defaults to False.

  • attention_num_heads (int, optional) – Number of heads. Defaults to 8.

  • num_enc_layers (int, optional) – Number of layers in the encoder. Defaults to 3.

  • enc_same (bool, optional) – Use the same encoder for all context sets. This only works if all context sets have the same dimensionality. Defaults to False.

  • num_dec_layers (int, optional) – Number of layers in the decoder. Defaults to 6.

  • width (int, optional) – Widths of all intermediate MLPs. Defaults to 512.

  • nonlinearity (Callable or str, optional) – Nonlinearity. Can also be specified as a string: “ReLU” or “LeakyReLU”. Defaults to ReLUs.

  • likelihood (str, optional) – Likelihood. Must be one of “het”, “lowrank”, “spikes-beta”, or “bernoulli-gamma”. Defaults to “lowrank”.

  • num_basis_functions (int, optional) – Number of basis functions for the low-rank likelihood. Defaults to 512.

  • dim_lv (int, optional) – Dimensionality of the latent variable. Defaults to 0.

  • lv_likelihood (str, optional) – Likelihood of the latent variable. Must be one of “het” or “dense”. Defaults to “het”.

  • transform (str or tuple[float, float]) – Bijection applied to the output of the model. This can help deal with positive of bounded data. Must be either “positive”, “exp”, “softplus”, or “softplus_of_square” for positive data or (lower, upper) for data in this open interval.

  • dtype (dtype, optional) – Data type.

Returns:

GNP model.

Return type:

model.Model

Attentive NPs#

neuralprocesses.architectures.agnp.construct_agnp(*args, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>, num_heads=8, **kw_args)[source]#

An Attentive Gaussian Neural Process.

Parameters:

  • dim_x (int, optional) – Dimensionality of the inputs. Defaults to 1.

  • dim_y (int, optional) – Dimensionality of the outputs. Defaults to 1.

  • dim_yc (int or tuple[int], optional) – Dimensionality of the outputs of the context set. You should set this if the dimensionality of the outputs of the context set is not equal to the dimensionality of the outputs of the target set. You should also set this if you want to use multiple context sets. In that case, set this equal to a tuple of integers indicating the respective output dimensionalities.

  • dim_yt (int, optional) – Dimensionality of the outputs of the target set. You should set this if the dimensionality of the outputs of the target set is not equal to the dimensionality of the outputs of the context set.

  • dim_embedding (int, optional) – Dimensionality of the embedding. Defaults to 128.

  • num_heads (int, optional) – Number of heads. Defaults to 8.

  • num_enc_layers (int, optional) – Number of layers in the encoder. Defaults to 3.

  • enc_same (bool, optional) – Use the same encoder for all context sets. This only works if all context sets have the same dimensionality. Defaults to False.

  • num_dec_layers (int, optional) – Number of layers in the decoder. Defaults to 6.

  • width (int, optional) – Widths of all intermediate MLPs. Defaults to 512.

  • nonlinearity (Callable or str, optional) – Nonlinearity. Can also be specified as a string: “ReLU” or “LeakyReLU”. Defaults to ReLUs.

  • likelihood (str, optional) – Likelihood. Must be one of “het”, “lowrank”, “spikes-beta”, or “bernoulli-gamma”. Defaults to “lowrank”.

  • num_basis_functions (int, optional) – Number of basis functions for the low-rank likelihood. Defaults to 512.

  • dim_lv (int, optional) – Dimensionality of the latent variable. Defaults to 0.

  • lv_likelihood (str, optional) – Likelihood of the latent variable. Must be one of “het” or “dense”. Defaults to “het”.

  • transform (str or tuple[float, float]) – Bijection applied to the output of the model. This can help deal with positive of bounded data. Must be either “positive”, “exp”, “softplus”, or “softplus_of_square” for positive data or (lower, upper) for data in this open interval.

  • dtype (dtype, optional) – Data type.

Returns:

AGNP model.

Return type:

model.Model

Convolutional NPs#

neuralprocesses.architectures.convgnp.construct_convgnp(dim_x=1, dim_y=1, dim_yc=None, dim_yt=None, dim_aux_t=None, points_per_unit=64, margin=0.1, likelihood='lowrank', conv_arch='unet', unet_channels=(64, 64, 64, 64, 64, 64), unet_kernels=5, unet_strides=2, unet_activations=None, unet_resize_convs=False, unet_resize_conv_interp_method='nearest', conv_receptive_field=None, conv_layers=6, conv_channels=64, num_basis_functions=64, dim_lv=0, lv_likelihood='het', encoder_scales=None, encoder_scales_learnable=True, decoder_scale=None, decoder_scale_learnable=True, aux_t_mlp_layers=(128, 128, 128), divide_by_density=True, epsilon=0.0001, transform=None, dtype=None, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>)[source]#

A Convolutional Gaussian Neural Process.

Sets the attribute receptive_field to the receptive field of the model.

Parameters:

  • dim_x (int, optional) – Dimensionality of the inputs. Defaults to 1.

  • dim_y (int, optional) – Dimensionality of the outputs. Defaults to 1.

  • dim_yc (int or tuple[int], optional) – Dimensionality of the outputs of the context set. You should set this if the dimensionality of the outputs of the context set is not equal to the dimensionality of the outputs of the target set. You should also set this if you want to use multiple context sets. In that case, set this equal to a tuple of integers indicating the respective output dimensionalities.

  • dim_yt (int, optional) – Dimensionality of the outputs of the target set. You should set this if the dimensionality of the outputs of the target set is not equal to the dimensionality of the outputs of the context set.

  • dim_aux_t (int, optional) – Dimensionality of target-specific auxiliary variables.

  • points_per_unit (float, optional) – Density of the internal discretisation. Defaults to 64.

  • margin (float, optional) – Margin of the internal discretisation. Defaults to 0.1.

  • likelihood (str, optional) – Likelihood. Must be one of “het”, “lowrank”, “spikes-beta”, or “bernoulli-gamma”. Defaults to “lowrank”.

  • conv_arch (str, optional) – Convolutional architecture to use. Must be one of “unet[-res][-sep]” or “conv[-res][-sep]”. Defaults to “unet”.

  • unet_channels (tuple[int], optional) – Channels of every layer of the UNet. Defaults to six layers each with 64 channels.

  • unet_kernels (int or tuple[int], optional) – Sizes of the kernels in the UNet. Defaults to 5.

  • unet_strides (int or tuple[int], optional) – Strides in the UNet. Defaults to 2.

  • unet_activations (object or tuple[object], optional) – Activation functions used by the UNet. If None, ReLUs are used.

  • unet_resize_convs (bool, optional) – Use resize convolutions rather than transposed convolutions in the UNet. Defaults to False.

  • unet_resize_conv_interp_method (str, optional) – Interpolation method for the resize convolutions in the UNet. Can be set to “bilinear”. Defaults to “nearest”.

  • conv_receptive_field (float, optional) – Receptive field of the standard architecture. Must be specified if conv_arch is set to “conv”.

  • conv_layers (int, optional) – Layers of the standard architecture. Defaults to 8.

  • conv_channels (int, optional) – Channels of the standard architecture. Defaults to 64.

  • num_basis_functions (int, optional) – Number of basis functions for the low-rank likelihood. Defaults to 512.

  • dim_lv (int, optional) – Dimensionality of the latent variable. Defaults to 0.

  • lv_likelihood (str, optional) – Likelihood of the latent variable. Must be one of “het” or “lowrank”. Defaults to “het”.

  • encoder_scales (float or tuple[float], optional) – Initial value for the length scales of the set convolutions for the context sets embeddings. Defaults to 1 / points_per_unit.

  • encoder_scales_learnable (bool, optional) – Whether the encoder SetConv length scale(s) are learnable.

  • decoder_scale (float, optional) – Initial value for the length scale of the set convolution in the decoder. Defaults to 1 / points_per_unit.

  • decoder_scale_learnable (bool, optional) – Whether the decoder SetConv length scale(s) are learnable.

  • aux_t_mlp_layers (tuple[int], optional) – Widths of the layers of the MLP for the target-specific auxiliary variable. Defaults to three layers of width 128.

  • divide_by_density (bool, optional) – Divide by the density channel. Defaults to True.

  • epsilon (float, optional) – Epsilon added by the set convolutions before dividing by the density channel. Defaults to 1e-4.

  • transform (str or tuple[float, float]) – Bijection applied to the output of the model. This can help deal with positive of bounded data. Must be either “positive”, “exp”, “softplus”, or “softplus_of_square” for positive data or (lower, upper) for data in this open interval.

  • dtype (dtype, optional) – Data type.

Returns:

ConvGNP model.

Return type:

model.Model

Fully Convolutional NPs#

neuralprocesses.architectures.fullconvgnp.construct_fullconvgnp(dim_x=1, dim_y=1, dim_yc=None, dim_yt=None, points_per_unit=64, margin=0.1, conv_arch='unet', unet_channels=(64, 64, 64, 64, 64, 64), unet_kernels=5, unet_strides=2, unet_activations=None, unet_resize_convs=False, unet_resize_conv_interp_method='nearest', conv_receptive_field=None, conv_layers=6, conv_channels=64, kernel_factor=2, dim_lv=0, encoder_scales=None, decoder_scale=None, divide_by_density=True, epsilon=0.0001, transform=None, dtype=None, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>)[source]#

A Fully Convolutional Gaussian Neural Process.

Sets the attribute receptive_field to the receptive field of the model.

Parameters:

  • dim_x (int, optional) – Dimensionality of the inputs. Defaults to 1.

  • dim_y (int, optional) – Dimensionality of the outputs. Defaults to 1.

  • dim_yc (int or tuple[int], optional) – Dimensionality of the outputs of the context set. You should set this if the dimensionality of the outputs of the context set is not equal to the dimensionality of the outputs of the target set. You should also set this if you want to use multiple context sets. In that case, set this equal to a tuple of integers indicating the respective output dimensionalities.

  • dim_yt (int, optional) – Dimensionality of the outputs of the target set. You should set this if the dimensionality of the outputs of the target set is not equal to the dimensionality of the outputs of the context set.

  • points_per_unit (float, optional) – Density of the internal discretisation. Defaults to 64.

  • margin (float, optional) – Margin of the internal discretisation. Defaults to 0.1.

  • conv_arch (str, optional) – Convolutional architecture to use. Must be one of “unet[-res][-sep]” or “conv[-res][-sep]”. Defaults to “unet”.

  • unet_channels (tuple[int], optional) – Channels of every layer of the UNet. Defaults to six layers each with 64 channels.

  • unet_kernels (int or tuple[int], optional) – Sizes of the kernels in the UNet. Defaults to 5.

  • unet_strides (int or tuple[int], optional) – Strides in the UNet. Defaults to 2.

  • unet_activations (object or tuple[object], optional) – Activation functions used by the UNet.

  • unet_resize_convs (bool, optional) – Use resize convolutions rather than transposed convolutions in the UNet. Defaults to False.

  • unet_resize_conv_interp_method (str, optional) – Interpolation method for the resize convolutions in the UNet. Can be set to “bilinear”. Defaults to “nearest”.

  • conv_receptive_field (float, optional) – Receptive field of the standard architecture. Must be specified if conv_arch is set to “conv”.

  • conv_layers (int, optional) – Layers of the standard architecture. Defaults to 8.

  • conv_channels (int, optional) – Channels of the standard architecture. Defaults to 64.

  • kernel_factor (int, optional) – Factor to reduce the number of channel of the kernel CNN architecture and the kernel points per unit by. Set to 1 to put the architecture for the kernel on equal footing with the architecture for the mean. Defaults to 2.

  • dim_lv (int, optional) – Dimensionality of the latent variable. Defaults to 0.

  • encoder_scales (float or tuple[float], optional) – Initial value for the length scales of the set convolutions for the context sets embeddings. Defaults to 1 / points_per_unit.

  • decoder_scale (float, optional) – Initial value for the length scale of the set convolution in the decoder. Defaults to 1 / points_per_unit.

  • divide_by_density (bool, optional) – Divide by the density channel. Defaults to True.

  • epsilon (float, optional) – Epsilon added by the set convolutions before dividing by the density channel. Defaults to 1e-4.

  • transform (str or tuple[float, float]) – Bijection applied to the output of the model. This can help deal with positive of bounded data. Must be either “positive”, “exp”, “softplus”, or “softplus_of_square” for positive data or (lower, upper) for data in this open interval.

  • dtype (dtype, optional) – Data type.

Returns:

FullConvGNP model.

Return type:

model.Model

Specific Models for Climate Experiments#

neuralprocesses.architectures.climate.construct_climate_convgnp_mlp(width_lr=128, lr_deg=0.75, likelihood='het', dtype=None, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>)[source]#

Construct a ConvGNP MLP model for climate downscaling.

References

  1. Vaughan, W. Tebbutt, J. S. Hosking, R. E. Turner. (2022). ``Convolutional

    Conditional Neural Processes for Local Climate Downscaling,’’ in Geoscientific Model Development 15(1), pages 251–268. URL: https://gmd.copernicus.org/articles/15/251/2022/

Parameters:

  • width_lr (int, optional) – Width of the low-resolution residual network. Defaults to 128.

  • lr_deg (float, optional) – Resolution of the low-resolution grid. Defaults to 0.75.

  • likelihood (str, optional) – Likelihood. Must be one of “het”, “lowrank”, “spikes-beta”, or “bernoulli-gamma”. Defaults to “lowrank”.

  • dtype (dtype, optional) – Data type.

neuralprocesses.architectures.climate.construct_climate_convgnp_multires(width_lr=128, width_mr=64, width_hr=64, width_bridge=64, lr_deg=0.75, mr_deg=0.1, hr_deg=0.01, mlp=False, likelihood='het', dtype=None, nps=<module 'neuralprocesses' from '/home/runner/work/neuralprocesses/neuralprocesses/neuralprocesses/__init__.py'>)[source]#

Construct a multi-resolution ConvGNP model for climate downscaling and fusion.

Parameters:

  • width_lr (int, optional) – Width of the low-resolution residual network. Defaults to 128.

  • width_mr (int, optional) – Width of the medium-resolution UNet. Defaults to 64.

  • width_hr (int, optional) – Width of the high-resolution UNet. Defaults to 64.

  • width_bridge (int, optional) – Number of channels to pass between the resolutions. Defaults to 64.

  • lr_deg (float, optional) – Resolution of the low-resolution grid. Defaults to 0.75.

  • mr_deg (float, optional) – Resolution of the medium-resolution grid. Defaults to 0.1.

  • hr_deg (float, optional) – Resolution of the high-resolution grid. Defaults to 0.01.

  • mlp (bool, optional) – Use an extra MLP at the end. Defaults to False.

  • likelihood (str, optional) – Likelihood. Must be one of “het” or “lowrank”. Defaults to `”lowrank”.

  • dtype (dtype, optional) – Data type.