{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tensors\n", "```\n", "Copyright 2022 National Technology & Engineering Solutions of Sandia,\n", "LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the\n", "U.S. Government retains certain rights in this software.\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Tensors are extensions of multidimensial arrays with additional operations defined on them. Here we explain the basics for creating and working with tensors." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pyttb as ttb\n", "import numpy as np\n", "import sys" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating a `tensor` from an array" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "M = np.ones((2, 4, 3)) # A 2x4x3 array.\n", "X = ttb.tensor(M) # Convert to a tensor object\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Optionally, you can specify a different shape for the `tensor`, so long as the input array has the right number of elements. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = X.reshape((4, 2, 3))\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating a one-dimensional `tensor`\n", "`np.random.rand(m,n)` creates a two-dimensional tensor with `m` rows and `n` columns." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(np.random.rand(5, 1)) # Creates a 2-way tensor.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To specify a 1-way `tensor`, use `(m,)` syntax, signifying a vector with `m` elements." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(np.random.rand(5), shape=(5,)) # Creates a 1-way tensor.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Specifying trailing singleton dimensions in a `tensor`\n", "Likewise, trailing singleton dimensions must be explicitly specified." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "Y = ttb.tensor(np.random.rand(4, 3)) # Creates a 2-way tensor.\n", "Y" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "Y = ttb.tensor(np.random.rand(3, 4, 1), (3, 4, 1)) # Creates a 3-way tensor.\n", "Y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The constituent parts of a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 4, 3)) # Create data.\n", "X.data # The array." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.shape # The shape." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating a `tensor` from its constituent parts" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 4, 3)) # Create data.\n", "Y = X.copy() # Copies X.\n", "Y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating an empty `tensor`\n", "An empty constructor exists." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = ttb.tensor() # Creates an empty tensor\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `tenones` to create a `tensor` of all ones" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = ttb.tenones((2, 3, 4)) # Creates a 2x3x4 tensor of ones.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `tenzeros` to create a `tensor` of all zeros" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = ttb.tenzeros((2, 1, 4)) # Creates a 2x1x4 tensor of zeroes.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `tenrand` to create a random `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 5, 4))\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `squeeze` to remove singleton dimensions from a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 5, 4)) # Create the data.\n", "Y = X.copy()\n", "# Add singleton dimension.\n", "Y[0, 0, 0, 0] = Y[0, 0, 0]\n", "# Remove singleton dimension.\n", "Y.squeeze().isequal(X)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `double` to convert a `tensor` to a (multidimensional) array" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 5, 4)) # Create the data.\n", "X.double() # Converts X to an array of doubles." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.data # Same thing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `ndims` and `shape` to get the shape of a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.ndims # Number of dimensions (or ways)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.shape # Row vector with the shapes of all dimensions." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.shape[2] # shape of a single dimension." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Subscripted reference for a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 3, 4, 1)) # Create a 3x4x2x1 random tensor.\n", "X[0, 0, 0, 0] # Extract a single element." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is possible to extract a subtensor that contains a single element. Observe that singleton dimensions are **not** dropped unless they are specifically specified, e.g., as above." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0, 0, 0, :] # Produces a tensor of order 1 and shape 1." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0, :, 0, :] # Produces a tensor of shape 3x1." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Moreover, the subtensor is automatically renumbered/resized in the same way that numpy works for arrays except that singleton dimensions are handled explicitly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0:2, 0, [1, 3], :] # Produces a tensor of shape 2x2x1." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's also possible to extract a list of elements by passing in an array of subscripts or a column array of linear indices." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "subs = np.array([[0, 0, 0, 0], [1, 2, 3, 0]])\n", "X[subs] # Extract 2 values by subscript." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "inds = np.array([0, 23])\n", "X[inds] # Same thing with linear indices." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((10,)) # Create a random tensor." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0:5] # Extract a subtensor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Subscripted assignment for a `tensor\n", "We can assign a single element, an entire subtensor, or a list of values for a `tensor`.`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 3, 4)) # Create some data.\n", "X[0, 0, 0] = 0 # Replaces the [0,0,0] element.\n", "X" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0, 0:2, 0:2] = np.ones((2, 2)) # Replaces a subtensor.\n", "X" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[(0, 0, 0)], X[1, 0, 0] = [5, 7] # Replaces the (0,0,0) and (1,0,0) elements." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[[0, 1]] = [5, 7] # Same as above using linear indices.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is possible to **grow** the `tensor` automatically by assigning elements outside the original range of the `tensor`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[2, 1, 1] = 1 # Grows the shape of the tensor.\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using negative indexing for the last array index" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tenrand((2, 3, 4)) # Create some data.\n", "np.prod(X.shape) - 1 # The index of the last element of the flattened tensor." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[2, 2, 3] = 99 # Inserting 99 into last element\n", "X[-1] # Same as X[2,2,3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X[0:-1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `find` for subscripts of nonzero elements of a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(3 * np.random.rand(2, 2, 2)) # Generate some data.\n", "X" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "S, V = X.find() # Find all the nonzero subscripts and values." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "S # Nonzero subscripts" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "V # Values" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "larger_entries = X >= 2\n", "larger_subs, larger_vals = larger_entries.find() # Find subscripts of values >= 2.\n", "larger_subs, larger_vals" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "V = X[larger_subs]\n", "V" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing the Frobenius norm of a `tensor`\n", "`norm` computes the Frobenius norm of a tensor. This corresponds to the Euclidean norm of the vectorized tensor." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(np.ones((3, 2, 3)))\n", "X.norm()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using `reshape` to rearrange elements in a `tensor`\n", "`reshape` reshapes a tensor into a given shape array. The total number of elements in the tensor cannot change." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(np.random.rand(3, 2, 3, 10))\n", "X.reshape((6, 30))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic operations (plus, minus, and, or, etc.) on a `tensor`\n", "`tensor`s support plus, minus, times, divide, power, equals, and not-equals operators. `tensor`s can use their operators with another `tensor` or a scalar (with the exception of equalities which only takes `tensor`s). All mathematical operators are elementwise operations." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "A = ttb.tensor(np.floor(3 * np.random.rand(2, 2, 3))) # Generate some data.\n", "B = ttb.tensor(np.floor(3 * np.random.rand(2, 2, 3)))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A.logical_and(B) # Calls and." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A.logical_or(B)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A.logical_xor(B)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A == B # Calls eq." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A != B # Calls neq." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A > B # Calls gt." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A >= B # Calls ge." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A < B # Calls lt." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A <= B # Calls le." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A.logical_not() # Calls not." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "+A # Calls uplus." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "-A # Calls uminus." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A + B # Calls plus." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A - B # Calls minus." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A * B # Calls times." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "5 * A # Calls mtimes." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A**B # Calls power." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A**2 # Calls power." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "A / B # Calls ldivide." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "2 / A # Calls rdivide." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using `tenfun` for elementwise operations on one or more `tensor`s\n", "The method `tenfun` applies a specified function to a number of `tensor`s. This can be used for any function that is not predefined for `tensor`s." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "A = ttb.tensor(np.floor(3 * np.random.rand(2, 2, 3), order=\"F\")) # Generate some data.\n", "A.tenfun(lambda x: x + 1) # Increment every element of A by one." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Wrap np.maximum in a function with a function signature that Python's inspect.signature can handle.\n", "def max_elements(a, b):\n", " return np.maximum(a, b)\n", "\n", "\n", "A.tenfun(max_elements, B) # Max of A and B, elementwise." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "C = ttb.tensor(\n", " np.floor(5 * np.random.rand(2, 2, 3), order=\"F\")\n", ") # Create another tensor.\n", "\n", "\n", "def elementwise_mean(X):\n", " # finding mean for the columns\n", " return np.floor(np.mean(X, axis=0), order=\"F\")\n", "\n", "\n", "A.tenfun(elementwise_mean, B, C) # Elementwise means for A, B, and C." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use `permute` to reorder the modes of a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = ttb.tensor(np.arange(1, 25), shape=(2, 3, 4))\n", "print(f\"X is a {X}\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X.permute(np.array((2, 1, 0))) # Reverse the modes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Permuting a 1-dimensional tensor works correctly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = ttb.tensor(np.arange(1, 5), (4,))\n", "X.permute(\n", " np.array(\n", " 1,\n", " )\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Symmetrizing and checking for symmetry in a `tensor`\n", "A `tensor` can be symmetrized in a collection of modes with the command `symmetrize`. The new, symmetric `tensor` is formed by averaging over all elements in the `tensor` which are required to be equal." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.rand(0)\n", "X = ttb.tensor(np.arange(1, 5), (4,)) # Create some data\n", "W = ttb.tensor(np.random.rand(4, 4, 4))\n", "Y = X.symmetrize()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An optional argument `grps` can also be passed to `symmetrize` which specifies an array of modes with respect to which the `tensor` should be symmetrized." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "X = ttb.tensor(np.random.rand(3, 3, 2))\n", "Z = X.symmetrize(np.array((0, 1)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Additionally, one can check for symmetry in tensors with the `issymmetric` function. Similar to `symmetrize`, a collection of modes can be passed as a second argument." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Y.issymmetric()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Z.issymmetric(np.array((1, 2)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Displaying a `tensor`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(X)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X # In the python interface" ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 1 }