Working with Big Data

Big Data My background is a bit unique in that I have had the opportunity to work intensively with three very different types of (very) big data: computational genomic data, retail transactional repositories, and capital market instrument pricing. Because of the very different characteristics of each of these types of data (except for the large scale nature of the datasets), I’ve had the opportunity to work on a variety of different approaches for highly efficient storage, retrieval, and analysis of the data. Particularly in the capital markets domain, the real-time nature of the datasets necessitated mission-critical procedures for manipulating the data. I’ve also had the opportunity to use highly specialized databases and languages (kdb+), as well as developing specialized proprietary database management systems for handling genomic data or retail transactional datasets.