S#

S#.Data (Hydra) - instruction manual and examples of use


S#.Data (Hydra) - instruction manual and examples of use
Atom
12/1/2016


Intro


Hydra is a program for downloading and storage of market data. This article will explain how to download the history from Google Finance, Gain Capital broker and IQFeed service.

To begin we tell a little bit about the program interface. After the first start you will see the main window, which prompts you to select the data sources.

image001.png


Pay attention to the description of each source, if the following is written next to it:

Source is designed to get history data … - then it means the ability to download historical data,

and if the following is written:

Source is designed to get market-data .. - then it means the ability to connect to the real-time data source and one's own history collection. Looking ahead, a similar process is often cheaper than a simple data purchase from a data vendor.

Now select the number of sources and try to get the market data. As such, we propose to use: Google (as a source of historical data), OECTrader (as a source of real-time data, which the broker gives you when opening an account), IQFeed (as a source of real-time data with the maximum number of concurrent subscriptions).

Click the check-boxes in the selected sources.

After you click OK, the program prompts you to enable additional features.

image003.png


They are well documented in the program itself, so we will not go into a description of each of them.

In order to move forward it will be enough just to click OK without selecting anything in this window. If necessary, you can always restore this setting via the ADD -> Tools button and select the required.

After all the previous procedure, we have got added sources in the left pane, all of which must now be set up.

Google Finance


This is done simply by pressing the pen button, image005.png

that opens the settings window.

image007.png


Quickly go over each of them to get a full understanding. So:


  • Start date – the date from which Hydra will receive market data
  • Time Offset – In this case, 1 means that the today data will not be downloaded. This is necessary in order not to download half a day, when the trading have not yet been completed.
  • Weekend – weekends ignored when checkbox is selected.
  • Time interval – Hydra downloads data in parts. This parameter allows to specify how large data part will be used. With a value of 30, the program will download data packages for 30 days.
  • Header, work from, work until, work interval – settings for the program itself, the time interval during which the program should download data (from and to)
  • Data directory – a directory in which the downloaded data will be stored, you can leave the default, or choose any other. We recommend to create a separate directory for each source, in order to not overwrite the data.
  • Format - the format of the stored data. BIN supported – Hydra’s special format allows to get a unique compression ratio (2 bytes per tick, 7 bytes per order book) or well-known CSV (it is normal volume)
  • Max.errors - the maximum number of errors in the source.
  • Dependency - specifies the added task that must be completed before the current (in this case, it may be either IQFeed, or OEC Trader)
  • Logging level - the level of logging.


Let's leave all the default data and choose the securities on which the data will be downloaded. Let's say we choose 5 tickers included in the S&P500 index, for example: MMM, AFL, GOOG, AAPl, T.

For this purpose, all these tools you need to add:

image009.png


Since Google source does not support the automatic addition of the tools you should to add them to the database manually

image011.png


Then repeat the procedure for each security and move them to the Selected section

image013.png


Then simply connect the source image015.png and press Start.

After the end of the downloading you have to get the following results

image017.png


OpenECry (Gain Futures)


The market data source for the Gain Capital clients, which a broker gives you for free when opening an account. The source key settings in many ways similar to the settings that we have made in Google with the only difference that now you should enter your account username/password to access the data stream and select the correct address from which the data will come. Preset 3 possibilities: the test server (to which we connect and which is intended for developers), the simulator (server for demo accounts), the real server (the server that has the connection to the real market and real accounts). If you have an opened account, then your choice is the real server.

image019.png


The CandleFromDate parameter is needed to specify the starting date from which you want to download the history in the form of candles.

Once it has been done, we add instruments by which we want to get data, because OEC supports automatic downloading and searching of instruments. We do it through the Download Securities button, and then adding them through the instrument code. Our selection ESZ6 (mini S&P) and NQZ6 (mini Nasdaq).

image022.gif


After adding you should get the same as shown in the picture.

Now turn your attention to the bottom part of the window. At the beginning of this article, we talked about the fact that Hydra supports downloading and storing a plurality of market data types, settings about what data to download can be done using the appropriate panel:

image023.png


On Ticks - means that ticks will be downloaded
On Market Depth - means that order books will be downloaded
Candles has additional setting, by clicking the button you can select the candles types which will be downloaded, that's how it looks:

image025.png


On Level 1 - means that the best bid/ask will be downloaded, as well as some fields of the ticker basic statistics.

Now just click the Start button, and Hydra will start to receive data and store them locally. In this case, both historical and real-time market data will be received.

Thus, you can collect and store the market data yourself, and then use them for testing your own strategies.

IQFeed


IQFeed provides as OpenECry, intraday data. But, in contrast to the OEC, IQFeed supports a very wide range of parallel subscriptions, as well as a much greater depth of history, both for candles and tick data.

The source settings similar to OpenEcry:

image027.png


The CandleFromDate parameter is needed to specify the starting date from which you want to download the history in the form of candles.

The TicksFromDate parameter is needed to specify the starting date from which you want to download the history in the form of ticks.

All other steps are similar to those described above.

Data viewing


Now, after all the manipulations, you need to understand, and how to work with these data, and what the program can do else. Recall that data is stored in the directory where you downloaded them, but how to see what is a result.

For this we refer to the OEC Trader source from which a lot of data has been downloaded, select it and click the right mouse button.

image029.png


Thereafter, we can choose in the menu the way to work with the instrument. Let's choose Market Depths and then in the window that opens, click the magnifying glass.
As a result, we will get a similar picture

image031.png


Next, similarly we select Level 1, and then Candles 1 min, but in the window with candles we do not choose the magnifying glass, and Chart button. The results are shown in the figures below

image033.png


image036.gif


That's the way you can view the data and work with them directly from the program.

We hope our product will help you in trading and development of profitable trading systems.



Thanks:




4 Answers
andros

Avatar
Date: 12/2/2016
Reply


Hi
I tried to connect to lmax on my account but i cannot download the data
please let me know how to download data from lmax
do i have to have a fix account to do that?
I have the basic Java/.net account subscription
Thanks:

Mikhail Sukhov

Avatar
Articles author Programmer Trader
Date: 12/2/2016
Reply


Could you please provide a log for LMAX source?

Yes we have FIX connectivity, but it will be as a paid component.
Topic starter
Thanks:

yuccafrance

Avatar
Date: 9/10/2017
Reply


Hello,

Is it possible to download data for the French market (board = EURONEXT Paris) by using Google or Yahoo Finance as sources ?

Thanks

Thanks:

Mikhail Sukhov

Avatar
Articles author Programmer Trader
Date: 9/11/2017
Reply


yuccafrance Перейти
Hello,

Is it possible to download data for the French market (board = EURONEXT Paris) by using Google or Yahoo Finance as sources ?

Thanks



Yahoo had deprecated their own service.

We recommend to use appropriate services. Like IQFeed or BarChart. Also you have an option via your broker in case his provide market data feed.

Topic starter
Thanks:


Attach files by dragging & dropping, , or pasting from the clipboard.

loading
clippy