Difference between revisions of "Q Trader Hints"
Jump to navigation
Jump to search
| Line 1: | Line 1: | ||
==Overview== | ==Overview== | ||
| − | In this project you will apply the Q-Learner you developed earlier to the trading problem. | + | In this project you will apply the Q-Learner you developed earlier to the trading problem. Note that there is no regression or classification learning in this project (so no use of RTLearner or LinRegLearner). The indicators define most of the "state" for your learner, an additional component of state you may use is whether or not you are currently holding a position long or short. The recommended actions are LONG, CASH, SHORT. |
Overall, your tasks for this project include: | Overall, your tasks for this project include: | ||
| Line 7: | Line 7: | ||
* Build a strategy learner based on your Q-Learner and previously developed indicators. | * Build a strategy learner based on your Q-Learner and previously developed indicators. | ||
* Test/debug the strategy learner on specific symbol/time period problems | * Test/debug the strategy learner on specific symbol/time period problems | ||
| − | |||
| − | |||
==Implement Strategy Learner== | ==Implement Strategy Learner== | ||
| Line 22: | Line 20: | ||
** Compute the reward for the last action | ** Compute the reward for the last action | ||
** Query the learner with the current state and reward to get an action | ** Query the learner with the current state and reward to get an action | ||
| − | ** Implement the action the learner returned ( | + | ** Implement the action the learner returned (LONG, CASH, SHORT), and update portfolio value |
* Repeat the above loop multiple times until cumulative return stops improving. | * Repeat the above loop multiple times until cumulative return stops improving. | ||
| − | |||
| − | |||
For the policy testing part: | For the policy testing part: | ||
| Line 31: | Line 27: | ||
** Compute the current state | ** Compute the current state | ||
** Query the learner with the current state to get an action | ** Query the learner with the current state to get an action | ||
| − | ** Implement the action the learner returned ( | + | ** Implement the action the learner returned (LONG, CASH, SHORT), and update portfolio value |
| − | * Return the resulting trades in a data frame | + | ** DO NOT UPDATE Q -- learning must be turned off in this phase |
| + | * Return the resulting trades in a data frame | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Training and testing for each situation should run in less than 30 seconds. We reserve the right to use different time periods if necessary to reduce auto grading time. | Training and testing for each situation should run in less than 30 seconds. We reserve the right to use different time periods if necessary to reduce auto grading time. | ||
Latest revision as of 10:58, 22 November 2017
Overview
In this project you will apply the Q-Learner you developed earlier to the trading problem. Note that there is no regression or classification learning in this project (so no use of RTLearner or LinRegLearner). The indicators define most of the "state" for your learner, an additional component of state you may use is whether or not you are currently holding a position long or short. The recommended actions are LONG, CASH, SHORT.
Overall, your tasks for this project include:
- Build a strategy learner based on your Q-Learner and previously developed indicators.
- Test/debug the strategy learner on specific symbol/time period problems
Implement Strategy Learner
For this part of the project you should develop a learner that can learn a trading policy using your Q-Learner. You should be able to use your Q-Learner from the earlier project directly, with no changes. You will need to write code in StrategyLearner.py to "wrap" your Q-Learner appropriately to frame the trading problem for it. Utilize the template provided in StrategyLearner.py Overall the structure of your strategy learner should be arranged as below. Note that this is a suggestion, not a requirement:
For the policy learning part:
- Select several technical features, and compute their values for the training data
- Discretize the values of the features
- Instantiate a Q-learner
- For each day in the training data:
- Compute the current state (including holding)
- Compute the reward for the last action
- Query the learner with the current state and reward to get an action
- Implement the action the learner returned (LONG, CASH, SHORT), and update portfolio value
- Repeat the above loop multiple times until cumulative return stops improving.
For the policy testing part:
- For each day in the testing data:
- Compute the current state
- Query the learner with the current state to get an action
- Implement the action the learner returned (LONG, CASH, SHORT), and update portfolio value
- DO NOT UPDATE Q -- learning must be turned off in this phase
- Return the resulting trades in a data frame
Training and testing for each situation should run in less than 30 seconds. We reserve the right to use different time periods if necessary to reduce auto grading time.
