Government website StandFor.co.nz published a word cloud representing comments received. Using Natural Language Processing we examine what they left out.
Referred to as the mullets of the internet, word clouds have some severe limitations.
Illustrative of these limitations, the largest term in the StandFor.co.nz word cloud, 'equality', appeared in 4.89% of comments compared to our analysis where the theme 'keeping the current flag' represented 31.96% of comments.
Read Matt Nippert's piece: Dissenting opinions discounted by flag panel.
StandFor.co.nz is a government-run website and was designed to inform the decision-making of the Flag Consideration Panel by soliciting comment from the public on shared New Zealand values.
Comment was allowed between the 5th of May and the 16th of July. Ten word clouds were published during this period.
Word clouds can miss multi-word terms, such as "new zealand", "tax payers" or "waste of money".
From a visualisation perspective, the importance of each word is hard to judge, as is the context or sentiment attached to it within a sentence.
The focus on words rather than concepts also has difficulty grouping similar ideas and naïve approaches misrepresent frequencies.
Further to that, this word cloud had the allowed pool of terms displayed filtered at the discretion of a human rather than a machine.
An alternative approach to represent the views of commenters is to use Natural Language Processing (NLP).
NLP is a field of computer science which includes technologies like machine learning and sentiment analysis and provides algorithms designed to group similar themes within text.
Using NLP tools developed by Entopix we produced this interactive to explore these themes.