mprod: Tensor - Tensor algebraic framework

mprod is a software implementation for tensor-tensor algebraic framework derived from the \(\star_{\bf{M}}\)-product 1. The package builds on NumPy2 and Scipy3 libraries to realize core operations and components required for the algebraic framework.

mprod-package implements the fundamental components required for the \(\star_{\mathbf{M}}\)-product algebraic framework; tensor-transpose, tensor-matrix multiplication (domain transforms), face-wise tensor multiplication, and, of course, the \(\star_{\mathbf{M}}\) tensor-tensor product (See intro)

In addition, the library offers several basic tensor factorizations such as mprod.decompostions.tsvdm 1 , and \(\star_{\mathbf{M}}\)-product based dimensionality reduction methods like the mprod.dimensionality_reduction.TCAM 4

An introductory cartoon for the TCAM 4 - an \(\star_{\mathbf{M}}\)-product based dimensionality reduction method for multi-way data.

You can find the software on github.

---

Installation

Conda install, with the great help of the conda-forge team:

conda install -c conda-forge mprod-package

The conda-forge packages are available for Linux, OS X, and Windows 64 bit. Local testing was done only on Linux.

PyPI install, presuming you have requirements installed (numpy, scipy, pandas, scikit-learn) installed:

pip install mprod-package

---

Scientific context

We live in a multi-dimensional world, immersed in huge volumes of data. This data often involves complex interlinked structures that span across multiple dimensions. Processes and phenomena also exhibit multi-dimensional behavior, requiring their models to operate in high dimensional settings.

Typically, we use matrix algebra to manipulate data, in so-called vector embedded spaces. But such representations usually don’t take into account the underlying integrity of an object’s dimension, either missing out on high-order links that go beyond pairwise relations or requiring an overhead in encoding such relations. This is where tensor algebra comes into play, addressing multiple dimensions.

But there is a problem. Despite a broad consensus, distilled over centuries of mathematical research, for matrix algebra, there is no such standard for its multidimensional counterpart, tensor algebra. There have been several propositions for tensor algebra frameworks over the years 5. Existing techniques that decompose tensor constructs into simpler tangible entities have limitations and inconsistencies compared to matrix algebra 6789. These issues have been hindering broad adoption of tensor algebra into mainstream use.

The tensor-tensor \(\star_{\bf{M}}\)-product framework aims to change that.

The paper “Tensor-Tensor Algebra for Optimal Representationand Compression of Multiway Data” 1 describes a way to bridge the gap between matrix and tensor algebra, resulting in new algebraic constructs that natively represent and manipulate high-dimensional entities, while preserving their multi-order integrity.

– Lior Horesh, IBM research 10

---

Contents

• Tutorials
  • TCAM
    • Minimal working example
      • Load files
      • Data normalization and mode-mapping
      • Transform
    • Supervised learning
      • The case of matrix valued inputs
      • ML workflows
      • TCAM as dimensionality reduction step
      • Concrete example
      • Dataset
      • Making the ML workflow
        • Model evaluation
  • ⚙ Background
    • The M-product
    • TSVDM
    • Tensor ranks and truncations
    • The effect of choosing different transforms
• API Reference
  • mprod base module
    • mprod.MeanDeviationForm
    • mprod.m_prod
    • mprod.tensor_mtranspose
    • mprod.x_m3
    • mprod.table2tensor
  • Dimensionality reductions
    • mprod.dimensionality_reduction.TCAM
  • Tensor decompositions
    • mprod.decompositions.svdm
    • mprod.decompositions.tqrm

---

Indices and tables

• Index

• Module Index

• Search Page

---

1(1,2,3)

Misha E. Kilmer, Lior Horesh, Haim Avron, and Elizabeth Newman. Tensor-tensor algebra for optimal representation and compression of multiway data. Proceedings of the National Academy of Sciences, 118(28):e2015851118, jul 2021. URL: https://www.pnas.org/content/118/28/e2015851118 https://www.pnas.org/content/118/28/e2015851118.abstract, doi:10.1073/PNAS.2015851118.

2

Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del R’ıo, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585(7825):357–362, September 2020. URL: https://doi.org/10.1038/s41586-020-2649-2, doi:10.1038/s41586-020-2649-2.

3

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, Aditya Vijaykumar, Alessandro Pietro Bardelli, Alex Rothberg, Andreas Hilboll, Andreas Kloeckner, Anthony Scopatz, Antony Lee, Ariel Rokem, C. Nathan Woods, Chad Fulton, Charles Masson, Christian Häggström, Clark Fitzgerald, David A. Nicholson, David R. Hagen, Dmitrii V. Pasechnik, Emanuele Olivetti, Eric Martin, Eric Wieser, Fabrice Silva, Felix Lenders, Florian Wilhelm, G. Young, Gavin A. Price, Gert-Ludwig Ingold, Gregory E. Allen, Gregory R. Lee, Hervé Audren, Irvin Probst, Jörg P. Dietrich, Jacob Silterra, James T Webber, Janko Slavič, Joel Nothman, Johannes Buchner, Johannes Kulick, Johannes L. Schönberger, José Vin’ıcius de Miranda Cardoso, Joscha Reimer, Joseph Harrington, Juan Luis Cano Rodr’ıguez, Juan Nunez-Iglesias, Justin Kuczynski, Kevin Tritz, Martin Thoma, Matthew Newville, Matthias Kümmerer, Maximilian Bolingbroke, Michael Tartre, Mikhail Pak, Nathaniel J. Smith, Nikolai Nowaczyk, Nikolay Shebanov, Oleksandr Pavlyk, Per A. Brodtkorb, Perry Lee, Robert T. McGibbon, Roman Feldbauer, Sam Lewis, Sam Tygier, Scott Sievert, Sebastiano Vigna, Stefan Peterson, Surhud More, Tadeusz Pudlik, Takuya Oshima, Thomas J. Pingel, Thomas P. Robitaille, Thomas Spura, Thouis R. Jones, Tim Cera, Tim Leslie, Tiziano Zito, Tom Krauss, Utkarsh Upadhyay, Yaroslav O. Halchenko, and Yoshiki Vázquez-Baeza and. SciPy 1.0: fundamental algorithms for scientific computing in python. Nature Methods, 17(3):261–272, February 2020. URL: https://doi.org/10.1038/s41592-019-0686-2, doi:10.1038/s41592-019-0686-2.

4(1,2)

Uria Mor, Yotam Cohen, Rafael Valdes-Mas, Denise Kviatcovsky, Eran Elinav, and Haim Avron. Dimensionality reduction of longitudinal ‘omics data using modern tensor factorization. 2021. arXiv:2111.14159.

5

Tamara G Kolda and Brett W Bader. Tensor Decompositions and Applications *. Society for Industrial and Applied Mathematics, 51(3):455–500, 2009. URL: http://three-mode.leidenuniv.nl/bibliogr/bibliogr.htm, doi:10.1137/07070111X.

6

Frank L. Hitchcock. The Expression of a Tensor or a Polyadic as a Sum of Products. Journal of Mathematics and Physics, 1927. doi:10.1002/sapm192761164.

7

Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, January 2000. URL: https://doi.org/10.1137/s0895479896305696, doi:10.1137/s0895479896305696.

8

I. V. Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):2295–2317, January 2011. URL: https://doi.org/10.1137/090752286, doi:10.1137/090752286.

9

L. R. Tucker. Implications of factor analysis of three-way matrices for measurement of change. In C. W. Harris, editor, Problems in measuring change., pages 122–137. University of Wisconsin Press, Madison WI, 1963.

10

Lior Horesh. New tensor algebra changes the rules of data analysis. URL: https://research.ibm.com/blog/new-tensor-algebra.