Project 8 Development

Due Date

4/19/2020 11:59PM Anywhere on Earth time

Revisions

This assignment is subject to change up until 3 weeks prior to the due date. We do not anticipate changes; any changes will be logged in this section.

Overview

In this project you will take a minimum of the 3 indicators created in Project 6 and:

• Implement a Manual Strategy (manual rule based trader) by:
  • Using your intuition and Technical Analysis, and test it against a stock using your market simulator
• Implement a Strategy Learner
  • You must draw on the learners you have created so far in the course. Your choices are:
  1. Regression or classification-based learner: Create a strategy using your Random Forest learner. Suggestions if you follow this approach: Classification_Trader_Hints. Important note, if you choose this method, you must set the leaf_size for your learner to 5 or greater. This is to avoid degenerate overfitting in-sample. For classification, you must use mode rather than mean (RTLearner, BagLearner).
  2. Reinforcement-based learner: Create a Q-learning-based strategy using your Q-Learner. Read the Classification_Trader_Hints first, because many of the ideas there are relevant for the Q trader, then see Q_Trader_Hints For Q-learning, use the same binning cuts for in-sample and out-of-sample.
  3. Optimization-based learner: Create a scan-based strategy using an optimizer. Read the Classification_Trader_Hints first, because many of the ideas there are relevant for the Opto trader, then see Opto_Trader_Hints
  • Regardless of your choice above, your learner should work in the following way:
    • In the training phase (e.g., addEvidence()) your learner will be provided with a stock symbol and a time period. It should use this data to learn a strategy. For instance, for a regression-based learner it will use this data to make predictions about future price changes.
    • In the testing phase (e.g., testPolicy()) your learner will be provided a symbol and a date range. All learning should be turned OFF during this phase.
    • You should use exactly the same indicators as you use in the Manual Strategy so we can compare your results. You may optimize your indicators for time (vectorization), but there should be no changes to lookback windows or any other pertinent parameters.
  • If the date range is the same as used for the training, it is an in-sample test. Otherwise it is an out-of-sample test. Your learner should return a trades dataframe like it did in the last project. Here are some important requirements: Your testPolicy() method should be much faster than your addEvidence() method. The timeout requirements (see rubric) will be set accordingly. Multiple calls to your testPolicy() method should return exactly the same result.

Overall, your tasks for this project include:

• Build a Manual Strategy that combines a minimum of 3 out of the 5 indicators from Project 6.
• Build a Strategy Learner based on one of the learners described above that uses the same 3+ indicators.
• Test/debug the Manual Strategy and Strategy Learner on specific symbol/time period problems.
• Write a report describing your Manual Strategy and Strategy Learner.

Template

Instructions:

• Download and install the files from this zip file File:20Spring strategy evaluation.zip
• ManualStrategy.py Code implementing a ManualStrategy object (your Manual Strategy) in the strategy_evaluation/ directory. It should implement testPolicy() which returns a trades data frame (see below). The main part of this code should call marketsimcode as necessary to generate the plots used in the report. NOTE: You will have to create this file yourself.
• Place your existing Q-Learner or RTLearner (Okay to include DTLearner as well if inheritance is involved) or OptimizationLearner into the strategy_evaluation/ directory.
• Place your existing indicators.py into the strategy_evaluation/ directory (NOTE: You can make changes to the indicators to properly work with both Manual Strategy and Strategy Learner but both strategies must use the same indicator code).
• Place your existing marketsimcode.py into the strategy_evaluation/ directory (optional: if needed).
• StrategyLearner.py Code implementing a StrategyLearner object (your ML strategy) in the strategy_evaluation directory.
• See "what to turn in" below for a list of files that should be submitted.
• To test your Strategy Learner, follow the instructions on Running the grading scripts

Data Details, Dates and Rules

• Use only the data provided for this course. You are not allowed to import external data.
• For your report, trade only the symbol JPM. This will enable us to more easily compare results. We will test your Strategy Learner with other symbols as well.
• You may use data from other symbols (such as SPY) to inform both your Manual Learner and Strategy Learner.
• The in-sample/development period is January 1, 2008 to December 31 2009.
• The out-of-sample/testing period is January 1, 2010 to December 31 2011.
• Starting cash is $100,000.
• Allowable positions are: 1000 shares long, 1000 shares short, 0 shares.
• Benchmark: The performance of a portfolio starting with $100,000 cash, investing in 1000 shares of the symbol in use and holding that position. Include transaction costs.
• There is no limit on leverage.
• Transaction costs:
  • ManualStrategy and StrategyLearner Report: Commission: $9.95, Impact: 0.005 (unless stated otherwise).
  • Auto-Grader Commission will always be $0.00, Impact may vary, and will be passed in as a parameter to the learner.
• Minimize use of herrings.

Tasks

Implement Manual Rule-Based Trader

Not included in template. You will have to create this code file

Create ManualStrategy.py and implement a set of rules using at a minimum of 3 indicators you created in Project 6 (NOTE: You can make changes to the indicators to properly work with both Manual Strategy and Strategy Learner but both strategies must use the same indicator code). Devise some simple logic using your indicators to enter and exit positions in the stock.

A recommended approach is to create a single logical expression that yields a -1, 0, or 1, corresponding to a "short," "out" or "long" position. Example usage is signal: If you are out of the stock, then a 1 would signal a BUY 1000 order. If you are long, a -1 would signal a SELL 2000 order. You don't have to follow this advice though, so long as you follow the trading rules outlined above.

For the report we want a written description, not code, however, it is OK to augment your written description with a pseudocode figure.

You should tweak your rules as best you can to get the best performance possible during the in-sample period (do not peek at out-of-sample performance). Use your rule-based strategy to generate a trades dataframe over the in-sample period.

We expect that your rule-based strategy should outperform the benchmark over the in-sample period.

Benchmark: The performance of a portfolio starting with $100,000 cash, investing in 1000 shares of JPM and holding that position.

Your ManualStrategy should implement the following API::

df_trades = ms.testPolicy(symbol = "AAPL", sd=dt.datetime(2010, 1, 1), ed=dt.datetime(2011,12,31), sv = 100000)

Implement Strategy Learner

For this part of the project you should develop a learner that can learn a trading policy using your learner and the same indicators used in ManualStrategy (NOTE: You can make changes to the indicators to properly work with both Manual Strategy and Strategy Learner but both strategies must use the same indicator code). You should be able to use your Q-Learner or RTLearner from the earlier project directly. If you want to use the optimization approach, you will need to create new code for that. You will need to write code in StrategyLearner.py to "wrap" your learner appropriately to frame the trading problem for it. Utilize the template provided in StrategyLearner.py.

Your Strategy Learner should find the optimal parameters for each indicator. Example: for SMA the learner should find the optimal loopback window to use.

Your StrategyLearner should implement the following API:

import StrategyLearner as sl
learner = sl.StrategyLearner(verbose = False, impact = 0.0, commission=0.0) # constructor
learner.addEvidence(symbol = "AAPL", sd=dt.datetime(2008,1,1), ed=dt.datetime(2009,12,31), sv = 100000) # training phase
df_trades = learner.testPolicy(symbol = "AAPL", sd=dt.datetime(2010,1,1), ed=dt.datetime(2011,12,31), sv = 100000) # testing phase

The input parameters are:

• verbose: if False do not generate any output
• impact: The market impact of each transaction
• commission: The commission amount charged.
• symbol: The stock symbol to train on
• sd: A datetime object that represents the start date
• ed: A datetime object that represents the end date
• sv: Start value of the portfolio

The output result is:

• df_trades: A data frame whose values represent trades for each day. Legal values are +1000.0 indicating a BUY of 1000 shares, -1000.0 indicating a SELL of 1000 shares, and 0.0 indicating NOTHING. Values of +2000 and -2000 for trades are also legal when switching from long to short or short to long so long as net holdings are constrained to -1000, 0, and 1000.

Implement Experiment 1

Not included in template. You will have to create this code file

Compare your Manual Strategy with your Strategy Learner in-sample trading JPM. Create a chart that shows:

• Value of the ManualStrategy portfolio (normalized to 1.0 at the start)
• Value of the StrategyLearner portfolio (normalized to 1.0 at the start)
• Value of the Benchmark portfolio (normalized to 1.0 at the start)

The code that implements this experiment and generates the relevant charts and data should be submitted as experiment1.py.

Implement Experiment 2

Not included in template. You will have to create this code file

Conduct an experiment with your StrategyLearner that shows how changing the value of impact should affect in-sample trading behavior (use at least two metrics). Trade JPM on the in-sample period with a commission of $0.00.

The code that implements this experiment and generates the relevant charts and data should be submitted as experiment2.py.

See the 'Report' section on Experiment 2 for more details.

Implement File

Not included in template. You will have to create this code file

Create run_project.py and implement the necessary calls (following each respective API) Manual Strategy.py, StrategyLearner.py, experiment1.py and experiment2.py with the appropriate parameters to run everything needed for the report in a single Python call:

PYTHONPATH=../:. python run_project.py

This is to have a single entry point to test your code against the report.

Implement author() function

Deduction if not implemented

You should implement a function called author() that returns your Georgia Tech user ID as a string. This is the ID you use to log into Canvas. It is not your 9 digit student number. Here is an example of how you might implement author():

 def author():
     return 'tb34' # replace tb34 with your Georgia Tech username.

Implementing this method correctly does not provide any points, but there will be a penalty for not implementing it.

Create README.txt

Deduction if not present or instructions to run your code is incorrect

Create a file named README.txt that describes:

• Each file you are submitting.
• Step-by-step instructions to run your code.

Report (30 Points)

• Word Count limit: 4000 words
• Page count limit: 10 pages
• Chart limit: 10 charts

Indicator Overview (4 Points)

Briefly describe the indicators you used to devise your Manual Strategy and Strategy Learner from the indicators you implemented in Project 6. You must use a minimum of 3 indicators of the 5 you implemented in Project 6.

• Include the parameters for each indicator that are optimized in both Manual Strategy and Strategy Learner.

Manual Strategy (6 Points)

Describe how you combined your indicators to create an overall signal. How do you decide to enter and exit your positions and why? Why do you believe (or not) that this is an effective strategy? Create a chart that shows, in-sample:

• Benchmark (starting with $100,000 cash, investing in 1000 shares of JPM and holding that position): Green line
• Performance of Manual Strategy: Red line
  • Both should be normalized to 1.0 at the start.
• Vertical blue lines indicating LONG entry points.
• Vertical black lines indicating SHORT entry points.

Compare the performance of your Manual Strategy versus the benchmark for the in-sample and out-of-sample time periods. Provide a chart.

Evaluate the performance of your strategy in the out-of-sample period. Note that you should not train or tweak your approach on this data. You should use the classification learned using the in-sample data only. Create a chart that shows:

• out-of-sample:
  • Benchmark (starting with $100,000 cash, investing in 1000 shares of JPM and holding that position): Green line
  • Performance of Manual Strategy: Red line
    • Both should be normalized to 1.0 at the start.
  • Vertical blue lines indicating LONG entry points.
  • Vertical black lines indicating SHORT entry points.

Create a table that summarizes the performance of the stock, and the Manual Strategy for both in-sample and out-of-sample periods. Explain WHY these differences occur.

• At minimum the table should include:
  • Cumulative return of the benchmark and Manual Strategy portfolio
  • STDEV of daily returns of benchmark and Manual Strategy portfolio
  • Mean of daily returns of benchmark and Manual Strategy portfolio

Strategy Learner (5 Points)

The centerpiece of this section should be the description of how you utilized your learner to determine trades:

• Describe the steps you took to frame the trading problem as a learning problem for your learner.
• Describe how you discretized (standardized) or otherwise adjusted your data. If not, tell us why not.

Experiment 1 (10 Points)

Manual Strategy / Strategy Learner

Describe your experiment in detail: Assumptions, parameter values and so on.

Describe the outcome of your experiment.

Would you expect this relative result every time with in-sample data? Explain why or why not.

Create a chart that shows:

• Value of the ManualStrategy portfolio (normalized to 1.0 at the start)
• Value of the StrategyLearner portfolio (normalized to 1.0 at the start)
• Value of the Benchmark portfolio (normalized to 1.0 at the start)

Experiment 2 (5 Points)

Strategy Learner

Provide a hypothesis regarding how changing the value of impact should affect in-sample trading behavior and results (provide at least two metrics).

Your descriptions should be stated clearly enough that an informed reader could reproduce the results without referencing your code.

What to turn in

Be sure to follow these instructions diligently! No zip files.

Submit the following files (only) via Canvas before the deadline:

• Project 8: Strategy Evaluation (Report)
  • Your report as report.pdf.
• Project 8: Strategy Evaluation (Code)
  • Your code as <Implemented Learner>.py, indicators.py, ManualStrategy.py, StrategyLearner.py, experiment1.py, experiment2.py and marketsimcode.py (optional if needed)
    • <Implemented Learner.py> refers to: QLearner.py, or RTLearner (okay to submit DTLearner if using inheritance) and BagLearner.py or OptimizeLearner.py.
  • README.txt

Make sure that you submit all code that is necessary for your software to run including plotting. If your code crashes because of a missing file, you may lose signficant points on the code section.

Do not submit any other files. Penalties will be assessed for additional files outside of the allowed ones as stated above.

Unlimited resubmissions are allowed up to the deadline for the project. When resubmitting, be sure to submit ALL files each time.

Rubric

Report (30 Points)

• Is the report especially well written (up to 2 point bonus)
• Does the strategy utilize a minimum of 3 indicators? (up to -30 points)
• Does the report description match the code? (up to -10 points)

Indicator Overview (4 Points)

• Are the indicators used in Manual Strategy and Strategy Learner briefly described (up to -4 points)

Manual Strategy (6 Points)

• Is the trading strategy described with clarity and in sufficient detail that someone else could reproduce it? (up to -4 points)
• Did the student use the correct symbol? (-2 points)
• Did the student use the date periods? (-2 points)
• Does the strategy obey holding constraints (-6 points)
• Does the provided chart(s) include:
  • Historic value of benchmark normalized to 1.0 with green line (-1 point)
  • Historic value of portfolio normalized to 1.0 with red line (-1 point)
  • Are the appropriate date ranges covered? (-2 points)
  • Are vertical lines, appropriately colored, included to indicate entries (-1 point)
• Does the manual trading system provide higher cumulative return than the benchmark over the in-sample time period? (-2 points if not)
• Are differences between the in-sample and out-of-sample performances appropriately explained (up to -4 points)
• Is the required table present and correct (-2 points)

Strategy Learner (5 Points)

• Is the method by which the learner is utilized to create a trading strategy described sufficiently clearly that an informed reader could reproduce the results without referencing your code? (up to -5 points)
• Did the student choose to use optimization based learning, and did their learning strategy beat the Manual Strategy (+1 point)

Experiment 1 (10 Points)

• Is the required experiment explained well? (up to -7 points)
• Is the required experiment compellingly supported the required chart? (-3 points)

Experiment 2 (5 Points)

• Is the required experiment explained well? (up to -3 points)
• Is the required experiment compellingly supported with tabular or graphical data?? (-2 points)

Code

• Does the submitted code indicators.py properly reflect the indicators provided in the report (up to -30 points if not)
• Are the indicators used for Manual Strategy and Strategy Learner the same as the ones used in Project 6? (up to -30 point if not)
• Does the submitted code and report reflect an understanding of the subject matter and follow the assignment directions? (up to -30 points if not)
• Missing author method. (up to -5 for each missing with max -10 points)
• Missing or incorrect instructions in the README.txt file (-10 points)
• Does the submitted code produce all included charts, with one run and within the time limit without any manipulation? (up to -30 points if not)
• Are the charts created and saved using Python code(DO NOT use plt.show() and manually save your charts)? (up to -30 points if not)

Auto-Grader (70 points)

We will test Strategy Learner in the following situations:

• Training / in-sample: January 1, 2008 to December 31 2009.
• Testing / out-of-sample: January 1, 2010 to December 31 2011.
• Symbols: ML4T-220, AAPL, UNH, SINE_FAST_NOISE
• Starting value: $100,000
• Benchmark: Buy 1000 shares on the first trading day, Sell 1000 shares on the last day.
• Commissions = $0.00, impact = 0.00

We expect the following outcomes in evaluating your system:

• For ML4T-220
  • addEvidence() completes without crashing within 25 seconds: 1 points
  • testPolicy() completes in-sample within 5 seconds: 2 points
  • testPolicy() returns same result when called in-sample twice: 2 points
  • testPolicy() returns an in-sample result with cumulative return greater than 100%: 5 points
  • testPolicy() returns an out-of-sample result with cumulative return greater than 100%: 5 points
• For AAPL
  • addEvidence() completes without crashing within 25 seconds: 1 points
  • testPolicy() completes in-sample within 5 seconds: 2 points
  • testPolicy() returns same result when called in-sample twice: 2 points
  • testPolicy() returns an in-sample result with cumulative return greater than benchmark: 5 points
  • testPolicy() returns an out-of-sample result within 5 seconds: 5 points
• For SINE_FAST_NOISE
  • addEvidence() completes without crashing within 25 seconds: 1 points
  • testPolicy() completes in-sample within 5 seconds: 2 points
  • testPolicy() returns same result when called in-sample twice: 2 points
  • testPolicy() returns an in-sample result with cumulative return greater than 200%: 5 points
  • testPolicy() returns an out-of-sample result within 5 seconds: 5 points
• For UNH
  • addEvidence() completes without crashing within 25 seconds: 1 points
  • testPolicy() completes in-sample within 5 seconds: 2 points
  • testPolicy() returns same result when called in-sample twice: 2 points
  • testPolicy() returns an in-sample result with cumulative return greater than benchmark: 5 points
  • testPolicy() returns an out-of-sample result within 5 seconds: 5 points
• Withheld test case 1: In-sample test case for an unknown symbol.
  • If any part of code crashes: 0 points awarded.
  • testPolicy() returns an in-sample result with cumulative return greater than benchmark: 5 points
• Withheld test case 2: In-sample test case to verify that strategy accounts for different values of impact
  • If any part of code crashes: 0 points awarded.
  • Learner returns different trades when impact value is significantly different: 5 points
• Is any submitted code lacking a comment with the student's name and User ID? -5 points for each occurance, up to a maximum of -20
• Does the submitted code and report reflect an understanding of the subject matter? (up to -50 if not)

We reserve the right to use different time periods if necessary to reduce auto grading time.

• IMPORTANT NOTES
  • For achieving the required cumulative return, recall that cr = (portval[-1]/portval[0]) - 1.0
  • The requirement that consecutive calls to testPolicy() produce the same output for the same input means that you cannot update, train, or tune your learner in this method. For example, a solution that uses Q-Learning should use querySetState() and not query() in testPolicy(). Updating, training, and tuning (query()) is fine inside addEvidence().
  • Your learner should not select different hyper-parameters based on the symbol. Hyper-parameters include (but are not limited to) things like features, discretization size, sub-learning methods (for ensemble learners). Tuning using cross-validation or otherwise pre-processing the data is OK, things like if symbol=="UNH" are not OK. There will be a withheld test case that checks your code on a valid symbol that is not one of the four listed above.
  • Presence of code like if symbol=="UNH" will result in a 20 point penalty.
  • When evaluating the trades generated by your learner, we will consider transaction costs (market impact).

Code & Report

Submission of code and report (up to 100 points deductions):

• Is the required code provided, including code to recreate the charts and usage of correct trades data frame. (up to -100 if not)
• Is the required report provided (-100 if not)