Edit this page

LogSoftMax

Versioned name : LogSoftmax-5

Category : Activation function

Short description : LogSoftmax computes the natural logarithm of softmax values for the given input.

Note : This is recommended to not compute LogSoftmax directly as Log(Softmax(x, axis)), more numeric stable is to compute LogSoftmax as:

$$\begin{array}{r}t=(x-ReduceMax(x,axis))\\ LogSoftmax(x,axis)=t-Log(ReduceSum(Exp(t),axis))\end{array}$$

Attributes

• axis

  • Description : axis represents the axis of which the LogSoftmax is calculated. Negative value means counting dimensions from the back.

  • Range of values : any integer value

  • Type : int

  • Default value : 1

  • Required : no

Inputs :

• 1 : Input tensor x of type T with enough number of dimension to be compatible with axis attribute. Required.

Outputs :

• 1 : The resulting tensor of the same shape and of type T.

Types

• T : any floating-point type.

Mathematical Formulation

$$y_c=ln\left(\frac{e^{Z_c}}{\sum _{d=1}^C{e}^{Z_d}}\right)$$

where $C$ is a size of tensor along axis dimension.

Example

<layer ... type="LogSoftmax" ... >
    <data axis="1" />
    <input>
        <port id="0">
            <dim>256</dim>
            <dim>56</dim>
        </port>
    </input>
    <output>
        <port id="3">
            <dim>256</dim>
            <dim>56</dim>
        </port>
    </output>
</layer>

Prev Next