List of Coders

Deep Sets

class neuralprocesses.coders.deepset.DeepSet(phi, agg=<function DeepSet.<lambda>>)

Deep set.

Parameters:

• phi (object) – Pre-aggregation function.

• agg (object, optional) – Aggregation function. Defaults to summing.

phi

Pre-aggregation function.

Type:

object

agg

Aggregation function.

Type:

object

class neuralprocesses.coders.inputs.InputsCoder

Encode with the target inputs.

Attention

class neuralprocesses.coders.attention.Attention(dim_x, dim_y, dim_embedding, num_heads, num_enc_layers, nonlinearity='ReLU', dtype=None, _self=False)

Attention module.

Parameters:

• dim_x (int) – Dimensionality of the inputs.

• dim_y (int) – Dimensionality of the outputs.

• dim_embedding (int) – Dimensionality of the embedding.

• num_heads (int) – Number of heads.

• num_enc_layers (int) – Number of layers in the encoders.

• nonlinearity (Callable or str, optional) – Nonlinearity. Can also be specified as a string: “ReLU” or “LeakyReLU”. Defaults to ReLUs.

• dtype (dtype, optional) – Data type.

num_heads

Number of heads.

Type:

int

dim_head

Dimensionality of a head.

Type:

int

encoder_x

Encoder for the inputs.

Type:

function

encoder_xy

Encoder for the inputs-output pairs.

Type:

function

mixer

Mixer.

Type:

function

mlp1

First MLP for the final normalisation layers.

Type:

function

ln1

First normaliser for the final normalisation layers.

Type:

function

mlp2

Second MLP for the final normalisation layers.

Type:

function

ln2

Second normaliser for the final normalisation layers.

Type:

function

class neuralprocesses.coders.attention.SelfAttention(*args, **kw_args)

Self-attention module.

Parameters:

• dim_x (int) – Dimensionality of the inputs.

• dim_y (int) – Dimensionality of the outputs.

• dim_embedding (int) – Dimensionality of the embedding.

• num_heads (int) – Number of heads.

• num_enc_layers (int) – Number of layers in the encoders.

• nonlinearity (string, optional) – Nonlinearity in the encoders. Valid

• None (values are) –

• case-sensitive. –

• dtype (dtype, optional) – Data type.

num_heads

Number of heads.

Type:

int

dim_head

Dimensionality of a head.

Type:

int

encoder_x

Encoder for the inputs.

Type:

function

encoder_xy

Encoder for the inputs-output pairs.

Type:

function

mixer

Mixer.

Type:

function

mlp1

First MLP for the final normalisation layers.

Type:

function

ln1

First normaliser for the final normalisation layers.

Type:

function

mlp2

Second MLP for the final normalisation layers.

Type:

function

ln2

Second normaliser for the final normalisation layers.

Type:

function

Convolutional Deep Sets

class neuralprocesses.coders.functional.FunctionalCoder(disc, coder, target=<function FunctionalCoder.<lambda>>)

A coder that codes to a discretisation for a functional representation.

Parameters:

• disc (discretisation.AbstractDiscretisation) – Discretisation.

• coder (coder) – Coder.

• target (function, optional) – Function which takes in the inputs of the current encoding and the desired inputs and which returns a tuple containing the inputs to span the discretisation over.

disc

Discretisation.

Type:

discretisation.AbstractDiscretisation

coder

Coder.

Type:

coder

target

Function which takes in the inputs of the current encoding and the desired inputs and which returns a tuple containing the inputs to span the discretisation over.

Type:

function

class neuralprocesses.coders.setconv.setconv.SetConv(scale, dtype=None, learnable=True)

A set convolution.

Parameters:

• scale (float) – Initial value for the length scale.

• dtype (dtype, optional) – Data type.

• learnable (bool, optional) – Whether the SetConv length scale is learnable.

log_scale

Logarithm of the length scale.

Type:

scalar

class neuralprocesses.coders.setconv.density.DivideByFirstChannel(epsilon: float = 1e-08)

Divide by the first channel.

Parameters:

epsilon (float) – Value to add to the channel before dividing.

epsilon

Value to add to the channel before dividing.

Type:

float

class neuralprocesses.coders.setconv.density.DivideByFirstHalf(epsilon: float = 1e-08)

Divide by the first half of channels.

Parameters:

epsilon (float) – Value to add to the channels before dividing.

epsilon

Value to add to the channels before dividing.

Type:

float

class neuralprocesses.coders.setconv.density.PrependDensityChannel(multi=False)

Prepend a density channel to the current encoding.

Parameters:

multi (bool, optional) – Produce a separate density channel for every channel. If False, produce just one density channel for all channels. Defaults to False.

class neuralprocesses.coders.setconv.density.PrependMultiDensityChannel

Prepend a separate density channel for every channel to the current encoding.

class neuralprocesses.coders.setconv.identity.PrependIdentityChannel

Prepend a density channel to the current encoding.

Neural Networks

class neuralprocesses.coders.nn.Conv(dim: int, in_channels: int, out_channels: int, kernel: Union[int, Tuple[int, ...]], stride: int = 1, transposed: bool = False, activation=None, separable: bool = False, residual: bool = False, dtype=None)

A flexible standard convolutional block.

Parameters:

• dim (int) – Dimensionality.

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• kernel (int or tuple[int]) – Kernel size(s). If it is a tuple, layers with those kernel sizes will be put in sequence.

• stride (int, optional) – Stride.

• transposed (bool, optional) – Transposed convolution. Defaults to False.

• separable (bool, optional) – Use depthwise separable convolutions. Defaults to False.

• residual (bool, optional) – Make a residual block. Defaults to False.

• dtype (dtype, optional) – Data type.

dim

Dimensionality.

Type:

int

net

Network.

Type:

object

class neuralprocesses.coders.nn.ConvNet(dim: int, in_channels: int, out_channels: int, channels: int, num_layers: int, kernel: Optional[int] = None, points_per_unit: Optional[float] = 1, receptive_field: Optional[float] = None, separable: bool = True, residual: bool = False, dtype=None)

A regular convolutional neural network.

Parameters:

• dim (int) – Dimensionality.

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• channels (int) – Number of channels at every intermediate layer.

• num_layers (int) – Number of layers.

• points_per_unit (float, optional) – Density of the discretisation corresponding to the inputs.

• receptive_field (float, optional) – Desired receptive field.

• kernel (int, optional) – Kernel size. If set, then this overrides the computation done by points_per_unit and receptive_field.

• separable (bool, optional) – Use depthwise separable convolutions. Defaults to True.

• dtype (dtype, optional) – Data type.

dim

Dimensionality.

Type:

int

kernel

Kernel size.

Type:

int

num_halving_layers

Number of layers with stride equal to two.

Type:

int

receptive_field

Receptive field.

Type:

float

conv_net

The architecture.

Type:

module

class neuralprocesses.coders.nn.Linear(in_channels, out_channels, dtype)

A linear layer over channels.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• dtype (dtype, optional) – Data type.

net

Linear layer.

Type:

object

class neuralprocesses.coders.nn.MLP(in_dim: int, out_dim: int, layers: Optional[Tuple[int, ...]] = None, num_layers: Optional[int] = None, width: Optional[int] = None, nonlinearity: Union[Callable, str] = 'ReLU', dtype=None)

MLP.

Parameters:

• in_dim (int) – Input dimensionality.

• out_dim (int) – Output dimensionality.

• layers (tuple[int, ...], optional) – Width of every hidden layer.

• num_layers (int, optional) – Number of hidden layers.

• width (int, optional) – Width of the hidden layers

• nonlinearity (Callable or str, optional) – Nonlinearity. Can also be specified as a string: “ReLU” or “LeakyReLU”. Defaults to ReLUs.

• dtype (dtype, optional) – Data type.

net

MLP, but which expects a different data format.

Type:

object

class neuralprocesses.coders.nn.ResidualBlock(layer1, layer2, layer_post)

Block of a residual network.

Parameters:

• layer1 (object) – Layer in the first branch.

• layer2 (object) – Layer in the second branch.

• layer_post (object) – Layer after adding the output of the two branches.

layer1

Layer in the first branch.

Type:

object

layer2

Layer in the second branch.

Type:

object

layer_post

Layer after adding the output of the two branches.

Type:

object

class neuralprocesses.coders.nn.UNet(dim: int, in_channels: int, out_channels: int, channels: Tuple[int, ...] = (8, 16, 16, 32, 32, 64), kernels: Union[int, Tuple[Union[int, Tuple[int, ...]], ...]] = 5, strides: Union[int, Tuple[int, ...]] = 2, activations: Union[None, object, Tuple[object, ...]] = None, separable: bool = False, residual: bool = False, resize_convs: bool = False, resize_conv_interp_method: str = 'nearest', dtype=None)

UNet.

Parameters:

• dim (int) – Dimensionality.

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• channels (tuple[int], optional) – Channels of every layer of the UNet. Defaults to six layers each with 64 channels.

• kernels (int or tuple[int], optional) – Sizes of the kernels. Defaults to 5.

• strides (int or tuple[int], optional) – Strides. Defaults to 2.

• activations (object or tuple[object], optional) – Activation functions.

• separable (bool, optional) – Use depthwise separable convolutions. Defaults to False.

• residual (bool, optional) – Make residual convolutional blocks. Defaults to False.

• resize_convs (bool, optional) – Use resize convolutions rather than transposed convolutions. Defaults to False.

• resize_conv_interp_method (str, optional) – Interpolation method for the resize convolutions. Can be set to “bilinear”. Defaults to “nearest”.

• dtype (dtype, optional) – Data type.

dim

Dimensionality.

Type:

int

kernels

Sizes of the kernels.

Type:

tuple[int]

strides

Strides.

Type:

tuple[int]

activations

Activation functions.

Type:

tuple[function]

num_halving_layers

Number of layers with stride equal to two.

Type:

int

receptive_fields

Receptive field for every intermediate value.

Type:

list[float]

receptive_field

Receptive field of the model.

Type:

float

before_turn_layers

Layers before the U-turn.

Type:

list[module]

after_turn_layers

Layers after the U-turn

Type:

list[module]

Aggregations

class neuralprocesses.coders.aggregate.RepeatForAggregateInputPairs(coder)

If the coder coder encounters an aggregate of target inputs, perform the coding operation for every pair of elements in the aggregate with the keyword arguments select_channel_i and select_channel_j set to the indices of the particular outputs selected in the pair of elements of the aggregate of inputs.

Parameters:

coder (coder) – Coder.

class neuralprocesses.coders.aggregate.RepeatForAggregateInputs(coder)

If the coder coder encounters an aggregate of target inputs, perform the coding operation for every element in the aggregate with the keyword argument select_channel set to the index of the particular output selected in the element of the aggregate of inputs.

Parameters:

coder (coder) – Coder.

class neuralprocesses.coders.aggregate.SelectFromChannels(size0, *sizes)

Select an output channel within a RepeatForAggregateInputs.

The channels dimension is supposed to be a concatenation of multiple blocks of channels, where selecting the i`th output is performed by selecting the `i`th element of every block. The elements of `sizes specify the lengths of the blocks.

If an element of sizes is a tuple, then it is assumed that the channels of that block must be further reshaped into that tuple. The selection will then happen on the _last_ unpacked dimension.

Parameters:

*sizes (int or tuple[int]) – Specification of the channel blocks.

class neuralprocesses.coders.aggregate.SelectFromDenseCovarianceChannels

Select a pair of output channels from a dense covariance within a RepeatForAggregateInputPairs.

Auxiliary Variables

class neuralprocesses.coders.augment.AssertNoAugmentation

Assert no augmentation of the target inputs.

class neuralprocesses.coders.augment.Augment(coder)

Concatenate the augmentation of the input to the encoding, and remove any augmentation of target inputs.

Parameters:

coder (coder) – Coder to run after the augmentation.

coder

Coder to run after the augmentation.

Type:

coder

Utility

class neuralprocesses.coders.shaping.AssertNoParallel

Assert exactly one element in parallel or not a parallel of elements.

class neuralprocesses.coders.shaping.AssertParallel(n)

Assert a parallel of n elements.

Parameters:

n (int) – Number of elements asserted in parallel.

n

Number of elements asserted in parallel.

Type:

int

class neuralprocesses.coders.shaping.Identity

Identity coder.

class neuralprocesses.coders.shaping.RestructureParallel(current, new)

Restructure a parallel of things.

Parameters:

• current (tuple) – Current structure.

• new (tuple) – New structure.

current

Current structure.

Type:

tuple

new

New structure.

Type:

tuple

class neuralprocesses.coders.shaping.Splitter(size0, *sizes)

Split a tensor into multiple tensors.

Parameters:

*sizes (int) – Size of every split

sizes

Size of every split

Type:

tuple[int]

class neuralprocesses.coders.shaping.SqueezeParallel

If there is a parallel of exactly one element, remove the parallel.

class neuralprocesses.coders.fuse.Fuse(set_conv)

In a parallel of two things, interpolate the first to the inputs of the second, and concatenate the result to the second.

Parameters:

set_conv (coder) – Set conv that should perform the interpolation.

set_conv

Set conv that should perform the interpolation.

Type:

coder

class neuralprocesses.coders.mapdiag.MapDiagonal(coder)

Map to the diagonal of the squared space.

Parameters:

coder (coder) – Coder to apply the mapped values to.

coder

Coder to apply the mapped values to.

Type:

coder

class neuralprocesses.coders.densecov.DenseCovariancePSDTransform

Multiply a dense covariance encoding by itself transposed to ensure that it is PSD.

class neuralprocesses.coders.densecov.FromDenseCovariance

Shape a dense covariance encoding into a regular encoding.

class neuralprocesses.coders.densecov.ToDenseCovariance

Shape a regular encoding into a dense covariance encoding.