# Library

**McTorch**

**Manifold optimization library for deep learning**

The software is available at Github.

McTorch is a python-based manifold optimization library for deep learning, which adds manifold optimization functionality to PyTorch. McTorch builds on top of PyTorch and supports all PyTorch functions in addition to manifold optimization. This is done to ensure researchers and developers using PyTorch can easily experiment with McTorch functions without writing a single line of extra code. It supports several manifolds including the Stiefel manifold, the positive definite manifold, and the Hyperbolic manifold. The optimizer can be selected among SGD, Adagrad and conjugate gradient (CG).

**SGDLibrary**

**Stochastic optimization algorithm library**

The software is available at Github.

The SGDLibrary is a pure-MATLAB library of a collection of stochastic optimization algorithms. Let be a smooth real-valued function on , where is the dimension. This solves an unconstrained minimization problem of the form,

See the detailed explanation of the stochastic optimization here.

#### Supported stochastic optimization algorithms

Here is the list of the supported optimization algorithms.

#### An example of sample codes

The SGDLibrary is also operable on GNU Octave (Free software compatible with many MATLAB scripts). Note that this SGDLibrary internally contains the GDLibrary.

**NMFLibrary**

The software is available at Github.

The NMFLibrary is a pure-MATLAB library of a collection of algorithms of non-negative matrix factorization (NMF). Nonnegative matrix factorization (NMF) is a fundamental problem for discovering nonnegative latent factors and/or performing dimensionality reduction. NMF has been successfully applied in diverse technical fields, such as pattern recognition, image/video analysis, text mining, bioinformatics and Web analysis because non-negativity of the obtained factors gives understandable interpretations of data of interest. NMF approximates a nonnegative matrix as a product of two nonnegative matrices and . More concretely, given , we calculate and with respect to the Euclidean distance as

where is usually chosen such that , that is, is approximately represented by the two low-rank matrices.

**ClassifierToolbox**

The software is available at Github.

The ClassifierToolbox is a pure-MATLAB toolbox for classifier. This package provides various tools for classification, e.g., image classification, face recognition, and related applications.

**SparseGDLibrary**

The software is available at Github.

The SparseGDLibrary is a pure-MATLAB library of a collection of unconstrained optimization algorithms for sparse modeling. This package includes various solvers such as

- APG (Accelerated gradient descent, i.e., Nesterov AGD)
- ISTA (Iterative shrinkage-thresholding algorithm)
- FISTA (Fast iterative shrinkage-thresholding algorithm)
- CD (Coordinate descent) for Lasso and Elastic Net
- ADMM (The alternating direction method of multipliers) for Lasso.

**RSOpt**

**Riemannian stochastic optimization library**

The software is available at Github.

Let be a smooth real-valued function on a Riemannian manifold . The target problem concerns a given model variable on , and is expressed as

where is the total number of the elements.This RSOpt package provides the MATLAB implementation codes dedicated to those **stochastic** algorithms above.

Note that various manifold algorithms on various manifolds are implemented in MATLAB toolbox Manopt. The RSOpt codes are compliant to Manopt. Also, please see here for more comprehensive explanation of optimization algorithms on matrix manifolds.

See additional detailed explanations here.