Coinmarket — service for data collection and market analysis
TimeAlpha-release 3 months Beta-release 5 months
TechnologiesWeb React.js, Node.js, Symfony, Doctrine (MySQL) Mobile React.js (mobile layout)
The project with which the customer came to us is an analytical service that allows users to monitor the capitalization of cryptocurrencies. The service demonstrates the dynamics and volume of trading, provides sufficient tools for analytics.
- Implement the ability to parse data of different formats from exchanges.
- Create a system capable of processing large massives of data and visualize it.
The service analyzes a huge amount of information: we connected about 30 exchanges and gave an opportunity for the manager to connect more of the exchanges in the administrative panel. For this, we analyzed the API of the stock exchanges and identified common templates for parsing theirs. We included these templates in the logic for adding new exchanges. So, if a stock exchange is suitable for automatic parsers (about them later), you can simply add the exchange to the admin panel.
Parsing exchanges - the most difficult and most interesting part of the work on the project. The main task was data processing without loss of speed; we implemented parallel parsing to achieve the result.
Exchanges have different API formats, as mentioned above, so we used different types of parsers:
- a simple parser – when the exchange gives all the actual tickers in one URL
- a double parser – when the list of tickers is given to another URL, the list of trading pairs from the first method is taken as a parameter
- multiple parser – when ticker for each currency pair is given on a separate request. Requests are executed at the same time in one script
- a custom parsers – when the data from the exchange does not fall under any of the existing parsers, we wrote a unique class of parser
The second task was the processing of data received from parsing. First, we identified general metrics for the currencies we monitor on the stock exchange, such as the exchange rate to the dollar, volume, circulation supply, etc. And after that, we implemented the analysis of pairs for each currency, calculating rates, price changes, volume, and less important statistics. Basic data were visualized in various charts.
In the next step, we optimized tables for large samples using special indexes for output on the frontend. Also at the frontend, we included the opportunity to choose fiat currency. We parse the exchange rates from the most popular currencies to the dollar. So, a user can switch the fiat currency to display and watch the data like prices, volumes, etc. in euro or yuan, for example.