Matrix factorization is a popular method in the machine learning toolbox. It has remained the popular method for many practical applications since the work of Daniel Lee and Sebastian Seung (Learning the parts of objects by non-negative matrix factorization, Nature, 401, 788—791,1999). However, prior to our work, Matrix factorization methods were developed only for a single data source. Real-world applications, in particular with the rise of big data, involve many data sources. Such sources are often correlated and interact. This work was the first to provide a principled matrix factorization and algebra to jointly factorize data matrices. The critical property of our method is that it discovers the shared information among multiple data sources, whilst preserving the individual information from each data domain.