What's in?

Objective:

Show that NLP is able to support categorization based on naturaly occuring text;
Provide an example of a structured multi-dimensional view on a document repository;
Provide an interactive web-based browser for exploratory analysis.

Motivation: Vast amount of data with a hidden structure (patterns) inside.

Method: Topic Modeling (LDA).

Software: Python backend (Gensim); HTML/JavaScript frontend (Google Compact Language Detector, Google Charts).

Language

Most of the iTunes apps have English-language descriptions. In our analysis we will focus exclusively on the English-language segment of the repository. This decision has been made deliberately with the purpose of simplification and easier interpretability of the results. It also allows us to employ language-specific techniques, e.g. lemmatization.

Categories

The predefined iTunes categories are limited to 22 general-purpose areas.

The chart below shows language distribution excluding English (66 languages). German, French and Spanish apps dominate. Thereafter, we do not take into consideration these applications.

Topics

Results of LDA model (produced with Gensim) for 66 topics: