Rapid Prototyping of a Text Mining Application for Cryptocurrency Market Intelligence

Abstract

Blockchain represents a technology for establishing a shared, immutable version of the truth between a network of participants that do not trust one another, and therefore has the potential to disrupt any financial or other industries that rely on third-parties to establish trust. Recent trends in computing including: prevalence of Free and Open Source Software (FOSS); easy access to High Performance Computing (HPC i.e. 'The Cloud'); and increasingly advanced analytics capabilities such as Natural Language Processing (NLP) and Machine Learning (ML) allow for rapidly prototyping applications for analysis of trends in the emergence of Blockchain technology. A scaleable proof-of-concept pipeline that lays the groundwork for analysis of multiple streams of semi-structured data posted on social media is demonstrated. Preliminary analysis and performance metrics are presented and discussed. Future work is described that will scale the system to cloud-based, real-time, analysis of multiple data streams, with Information Extraction (IE) (ex. sentiment analysis) and Machine Learning capability.