Alternating least squares for Tucker model
Copyright 2022 National Technology & Engineering Solutions of Sandia,
LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
U.S. Government retains certain rights in this software.
The function pyttb.tucker_als() computes the best rank-\((R_1,R_2,\ldots,R_n)\) approximation of tensor \(\mathcal{X}\), according to the specified dimensions in the vector \((R_1,R_2,\ldots,R_n)\). The form of \(\mathcal{X}\) can be a tensor, sptensor, ktensor, or ttensor. The result is a ttensor.
The method is originally from Tucker (1966) and later revisited in De Lathauwer et al. (2000).
L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31:279-311, 1966, http://dx.doi.org/10.1007/BF02289464
L. De Lathauwer, B. De Moor, J. Vandewalle, On the best rank-1 and rank-(R_1, R_2, R_N) approximation of higher-order tensors, SIAM J. Matrix Analysis and Applications, 21:1324-1342, 2000, http://doi.org/10.1137/S0895479898346995
Note: Oftentimes it’s better to use pyttb.hosvd() instead.
import os
import sys
import pyttb as ttb
import numpy as np
Create a data tensor of shape (5, 4, 3)
np.random.seed(0) # Set seed for reproducibility
X = ttb.sptenrand(
[5, 4, 3], nonzeros=10
) # Create a tensor with 10 nonzeros using the 'nonzeros' param.
X
sparse tensor of shape (5, 4, 3) with 10 nonzeros and order F
[0, 0, 2] = 0.26455561210462697
[0, 2, 0] = 0.7742336894342167
[1, 3, 1] = 0.45615033221654855
[2, 1, 1] = 0.5684339488686485
[2, 2, 1] = 0.018789800436355142
[2, 3, 0] = 0.6176354970758771
[2, 3, 2] = 0.6120957227224214
[3, 1, 2] = 0.6169339968747569
[3, 3, 2] = 0.9437480785146242
[4, 2, 1] = 0.6818202991034834
Create an approximation with all ranks equal to 2
T = ttb.tucker_als(X, 2) # best rank(2,2,2) approximation
T
Tucker Alternating Least-Squares:
Iter 0: fit = 3.447071e-01 fitdelta = 3.4e-01
Iter 1: fit = 3.633748e-01 fitdelta = 1.9e-02
Iter 2: fit = 3.666282e-01 fitdelta = 3.3e-03
Iter 3: fit = 3.689686e-01 fitdelta = 2.3e-03
Iter 4: fit = 3.706438e-01 fitdelta = 1.7e-03
Iter 5: fit = 3.717753e-01 fitdelta = 1.1e-03
Iter 6: fit = 3.725083e-01 fitdelta = 7.3e-04
Iter 7: fit = 3.729702e-01 fitdelta = 4.6e-04
Iter 8: fit = 3.732561e-01 fitdelta = 2.9e-04
Iter 9: fit = 3.734310e-01 fitdelta = 1.7e-04
Iter 10: fit = 3.735372e-01 fitdelta = 1.1e-04
Iter 11: fit = 3.736015e-01 fitdelta = 6.4e-05
(Tensor of shape: (5, 4, 3)
Core is a
tensor of shape (2, 2, 2) with order F
data[:, :, 0] =
[[ 1.27279000e+00 -1.67070163e-03]
[ 1.01465015e-04 2.79556305e-01]]
data[:, :, 1] =
[[-0.15850994 0.00097856]
[ 0.00311484 0.72168356]]
U[0] =
[[ 5.54196607e-04 9.98323007e-01]
[ 3.62027807e-02 -5.10572842e-06]
[ 5.43516219e-01 1.94246971e-03]
[ 8.38617413e-01 -1.92351491e-03]
[ 7.34446743e-05 5.78247452e-02]]
U[1] =
[[-5.54213670e-05 -5.63307471e-04]
[ 4.16133877e-01 -1.66331412e-03]
[ 2.36213646e-03 9.99996772e-01]
[ 9.09300288e-01 -1.83657557e-03]]
U[2] =
[[ 0.35603206 0.93309453]
[ 0.11412179 0.01048844]
[ 0.92747905 -0.35947823]],
[None,
array([[0.3595079 , 0.43703195],
[0.6976312 , 0.06022547],
[0.66676672, 0.67063787],
[0.21038256, 0.1289263 ]]),
array([[0.31542835, 0.36371077],
[0.57019677, 0.43860151],
[0.98837384, 0.10204481]])],
{'params': (0.0001, 1000, 1, array([0, 1, 2])),
'iters': 11,
'normresidual': np.float64(1.2038024607325746),
'fit': np.float64(0.3736014755013256)})
Create an approximation with specific ranks of [2, 2, 1]
T = ttb.tucker_als(X, [2, 2, 1]) # best rank(2,2,1) approximation
T
Tucker Alternating Least-Squares:
Iter 0: fit = 9.473514e-02 fitdelta = 9.5e-02
Iter 1: fit = 2.296442e-01 fitdelta = 1.3e-01
Iter 2: fit = 2.670080e-01 fitdelta = 3.7e-02
Iter 3: fit = 2.755574e-01 fitdelta = 8.5e-03
Iter 4: fit = 2.770953e-01 fitdelta = 1.5e-03
Iter 5: fit = 2.773711e-01 fitdelta = 2.8e-04
Iter 6: fit = 2.774238e-01 fitdelta = 5.3e-05
(Tensor of shape: (5, 4, 3)
Core is a
tensor of shape (2, 2, 1) with order F
data[:, :, 0] =
[[ 1.27097122e+00 -2.09211686e-05]
[-3.74534404e-05 3.86751630e-01]]
U[0] =
[[ 3.03244294e-04 9.82053530e-01]
[ 4.42778953e-02 1.49740989e-04]
[ 6.01599455e-01 6.18051935e-03]
[ 7.97569723e-01 -5.06514409e-03]
[ 9.11526161e-05 1.88432980e-01]]
U[1] =
[[ 5.74845811e-05 6.08196916e-01]
[ 3.89880068e-01 -6.10988633e-03]
[ 1.26736143e-03 7.93761336e-01]
[ 9.20864769e-01 1.45643367e-03]]
U[2] =
[[0.3786518 ]
[0.13177771]
[0.91610996]],
[None,
array([[0.20887676, 0.16130952],
[0.65310833, 0.2532916 ],
[0.46631077, 0.24442559],
[0.15896958, 0.11037514]]),
array([[0.65632959],
[0.13818295],
[0.19658236]])],
{'params': (0.0001, 1000, 1, array([0, 1, 2])),
'iters': 6,
'normresidual': np.float64(1.3886352427479627),
'fit': np.float64(0.27742375057545454)})
Use a different ordering of the dimensions
T = ttb.tucker_als(X, 2, dimorder=[2, 1, 0])
T
Tucker Alternating Least-Squares:
Iter 0: fit = 3.024840e-01 fitdelta = 3.0e-01
Iter 1: fit = 3.647849e-01 fitdelta = 6.2e-02
Iter 2: fit = 3.679934e-01 fitdelta = 3.2e-03
Iter 3: fit = 3.699743e-01 fitdelta = 2.0e-03
Iter 4: fit = 3.713304e-01 fitdelta = 1.4e-03
Iter 5: fit = 3.722231e-01 fitdelta = 8.9e-04
Iter 6: fit = 3.727917e-01 fitdelta = 5.7e-04
Iter 7: fit = 3.731461e-01 fitdelta = 3.5e-04
Iter 8: fit = 3.733639e-01 fitdelta = 2.2e-04
Iter 9: fit = 3.734966e-01 fitdelta = 1.3e-04
Iter 10: fit = 3.735769e-01 fitdelta = 8.0e-05
(Tensor of shape: (5, 4, 3)
Core is a
tensor of shape (2, 2, 2) with order F
data[:, :, 0] =
[[ 1.27270552e+00 -1.98145584e-03]
[ 1.79435530e-04 2.80015614e-01]]
data[:, :, 1] =
[[-0.15918278 0.00117904]
[ 0.00329255 0.72142572]]
U[0] =
[[ 5.47518964e-04 9.98369702e-01]
[ 3.62028131e-02 -6.01571747e-06]
[ 5.43516192e-01 1.91824360e-03]
[ 8.38617434e-01 -1.89973928e-03]
[ 7.27790758e-05 5.70144694e-02]]
U[1] =
[[-6.97808323e-05 6.22841697e-07]
[ 4.16133442e-01 -1.93714805e-03]
[ 2.62134849e-03 9.99996131e-01]
[ 9.09299776e-01 -1.99629018e-03]]
U[2] =
[[ 0.35642397 0.9320929 ]
[ 0.11587932 0.02443826]
[ 0.92711053 -0.36139396]],
[array([[0.60484552, 0.73926358],
[0.03918779, 0.28280696],
[0.12019656, 0.2961402 ],
[0.11872772, 0.31798318],
[0.41426299, 0.0641475 ]]),
array([[0.36872517, 0.82099323],
[0.09710128, 0.83794491],
[0.09609841, 0.97645947],
[0.4686512 , 0.97676109]]),
None],
{'params': (0.0001, 1000, 1, array([2, 1, 0])),
'iters': 10,
'normresidual': np.float64(1.2038496593897914),
'fit': np.float64(0.37357691568341256)})
Use the "nvecs" initialization method
This initialization is more expensive but generally works very well.
T = ttb.tucker_als(X, 2, dimorder=[0, 1, 2], init="nvecs")
T
Computing 2 leading e-vector for factor 1.
Computing 2 leading e-vector for factor 2.
Tucker Alternating Least-Squares:
Iter 0: fit = 3.431977e-01 fitdelta = 3.4e-01
Iter 1: fit = 3.452953e-01 fitdelta = 2.1e-03
Iter 2: fit = 3.454463e-01 fitdelta = 1.5e-04
Iter 3: fit = 3.454585e-01 fitdelta = 1.2e-05
/home/docs/checkouts/readthedocs.org/user_builds/pyttb/envs/392/lib/python3.9/site-packages/pyttb/tensor.py:2024: ComplexWarning: Casting complex values to real discards the imaginary part
self.data[actualIdx] = value
(Tensor of shape: (5, 4, 3)
Core is a
tensor of shape (2, 2, 2) with order F
data[:, :, 0] =
[[ 1.28134569 -0.00423492]
[-0.01678659 0.10347465]]
data[:, :, 1] =
[[-0.05410898 0.03494158]
[ 0.01336928 0.67364048]]
U[0] =
[[-2.25372685e-04 -2.31002840e-03]
[ 3.60504385e-02 -8.44144689e-03]
[ 5.42954195e-01 4.24222481e-02]
[ 8.38872743e-01 -4.36495907e-02]
[ 1.39132432e-02 9.98107445e-01]]
U[1] =
[[-3.81204622e-05 2.66340731e-06]
[ 4.16865412e-01 4.75019433e-02]
[ 6.11161335e-03 9.98464506e-01]
[ 9.08947675e-01 -2.84990509e-02]]
U[2] =
[[ 0.23592519 -0.03977318]
[ 0.15389222 0.98808283]
[ 0.95950846 -0.14869568]],
[None,
array([[ 4.38558455e-16+0.j, -5.96188401e-16+0.j],
[ 3.85357475e-01+0.j, 1.35152399e-02-0.j],
[ 3.98308926e-03+0.j, 9.99859058e-01-0.j],
[ 9.22758772e-01+0.j, -9.96005336e-03+0.j]]),
array([[-0. , -0. ],
[-0.38920902, 0.92114947],
[ 0.92114947, 0.38920902]])],
{'params': (0.0001, 1000, 1, array([0, 1, 2])),
'iters': 3,
'normresidual': np.float64(1.2578871401165317),
'fit': np.float64(0.34545851644517134)})
Specify the initial guess manually
U0 = [np.random.rand(5, 2), np.random.rand(4, 2), np.random.rand(3, 2)]
T = ttb.tucker_als(X, 2, dimorder=[0, 1, 2], init=U0)
T
Tucker Alternating Least-Squares:
Iter 0: fit = 2.537626e-01 fitdelta = 2.5e-01
Iter 1: fit = 3.610897e-01 fitdelta = 1.1e-01
Iter 2: fit = 3.663016e-01 fitdelta = 5.2e-03
Iter 3: fit = 3.689843e-01 fitdelta = 2.7e-03
Iter 4: fit = 3.707288e-01 fitdelta = 1.7e-03
Iter 5: fit = 3.718567e-01 fitdelta = 1.1e-03
Iter 6: fit = 3.725713e-01 fitdelta = 7.1e-04
Iter 7: fit = 3.730148e-01 fitdelta = 4.4e-04
Iter 8: fit = 3.732861e-01 fitdelta = 2.7e-04
Iter 9: fit = 3.734505e-01 fitdelta = 1.6e-04
Iter 10: fit = 3.735496e-01 fitdelta = 9.9e-05
(Tensor of shape: (5, 4, 3)
Core is a
tensor of shape (2, 2, 2) with order F
data[:, :, 0] =
[[1.27358153e+00 5.78083962e-04]
[9.26654182e-04 2.66531368e-01]]
data[:, :, 1] =
[[-0.15202347 -0.00191261]
[-0.00177027 0.72643245]]
U[0] =
[[-8.37927959e-04 9.97421314e-01]
[ 3.62038232e-02 2.62791661e-04]
[ 5.43519078e-01 -2.66961465e-03]
[ 8.38615271e-01 2.72772405e-03]
[ 1.43322311e-04 -7.16664969e-02]]
U[1] =
[[ 6.72753212e-05 9.52629611e-04]
[ 4.16135369e-01 1.39137514e-03]
[ 1.55068156e-04 9.99998252e-01]
[ 9.09302659e-01 -8.07357563e-04]]
U[2] =
[[ 0.35119109 0.93419866]
[ 0.10031332 -0.10417526]
[ 0.93091464 -0.3412043 ]],
[array([[0.69247212, 0.56660145],
[0.26538949, 0.52324805],
[0.09394051, 0.5759465 ],
[0.9292962 , 0.31856895],
[0.66741038, 0.13179786]]),
array([[0.7163272 , 0.28940609],
[0.18319136, 0.58651293],
[0.02010755, 0.82894003],
[0.00469548, 0.67781654]]),
array([[0.27000797, 0.73519402],
[0.96218855, 0.24875314],
[0.57615733, 0.59204193]])],
{'params': (0.0001, 1000, 1, array([0, 1, 2])),
'iters': 10,
'normresidual': np.float64(1.2039021505574306),
'fit': np.float64(0.37354960190807895)})