TransformerEncoderLayer#

class deeplay.components.transformer.enc.TransformerEncoderLayer(*args, **kwargs)#

Bases: DeeplayModule

Transformer encoder module.

Configurables#

  • in_features (int): Number of input features. If None, the input shape is inferred in the first forward pass. (Default: None)

  • hidden_features (list[int]): Number of hidden units in each layer.

  • out_features (int): Number of output features.

  • num_heads (int): Number of attention heads.

Shorthands#

  • input: Equivalent to .blocks[0].

  • hidden: Equivalent to .blocks[:-1].

  • output: Equivalent to .blocks[-1].

  • multihead: Equivalent to .blocks.multihead.

  • feed_forward: Equivalent to .blocks.feed_forward.

Evaluation#

>>> for block in tel.blocks:
>>>    x = block(x)
>>> return x

Examples#

>>> tel = TransformerEncoderLayer(4, [4, 16], 4, 2)
>>> tel.build()
>>> seq_len, batch_size,features = 2, 10, 4
>>> input_seq = torch.randn(seq_len, batch_size, features)
>>> tel(input_seq).shape
torch.Size([2, 10, 4])

Return Values#

The forward method returns the processed tensor.

Attributes Summary

feed_forward

Return the feed forward layer of the network.

hidden

Return the hidden layers of the network.

input

Return the input layer of the network.

multihead

Return the multihead attention layer of the network.

output

Return the last layer of the network.

Methods Summary

configure(*args, **kwargs)

Configures the module with specified arguments.

forward(x)

Define the computation performed at every call.

Attributes Documentation

feed_forward#

Return the feed forward layer of the network. Equivalent to .blocks.feed_forward.

hidden#

Return the hidden layers of the network. Equivalent to .blocks[:-1]

input#

Return the input layer of the network. Equivalent to .blocks[0].

multihead#

Return the multihead attention layer of the network. Equivalent to .blocks.multihead.layer.

output#

Return the last layer of the network. Equivalent to .blocks[-1].

Methods Documentation

configure(*args: Any, **kwargs: Any)#

Configures the module with specified arguments.

This method allows dynamic configuration of the module’s properties and behaviors. It can be used to set or modify the attributes and parameters of the module and, if applicable, its child modules. The method intelligently handles both direct attribute configuration and delegation to child modules’ configure methods.

Parameters#

*argsAny

Positional arguments specifying the configuration settings. When the first argument is a string matching a configurable attribute, the method expects either one or two arguments: the attribute name and, optionally, its value. If the attribute is itself a DeeplayModule, subsequent arguments are passed to its configure method.

**kwargsAny

Keyword arguments for configuration settings. If provided, these are used to update the module’s configuration directly.

Raises#

ValueError

Raised if a configuration key is not recognized as a valid configurable for the module or if the provided arguments do not match the expected pattern for configuration.

Example Usage#

To configure a single attribute: ` module.configure('attribute_name', attribute_value) # or module.configure(attribute_name=attribute_value) `

To configure multiple attributes using keyword arguments: ` module.configure(attribute1=value1, attribute2=value2) `

To configure a child module’s attribute: ` module.configure('child_module_attribute', child_attribute=child_attribute_value) # or module.child_module.configure(child_attribute=child_attribute_value) `

forward(x)#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.