Difference between revisions of "Manipulating Financial Data in Python"

From Quantitative Analysis Software Courses
Jump to navigation Jump to search
Line 26: Line 26:
  
 
Reading: "Python for Finance", Chapter 4: Data types and structures
 
Reading: "Python for Finance", Chapter 4: Data types and structures
 
<b>script</B>
 
*Numpy relationship to Pandas
 
*Creating Arrays
 
**empty, zeros, ones
 
*Basic Indexing and Slicing (start at 0 not 1)
 
*[quiz: print 2nd & 3rd columns]
 
*Index one array by another
 
*Reshaping
 
*Data Processing using Arrays
 
**Sum rows, Sum columns
 
**Statistics on columns: Mean, Median, stddev
 
*See: http://wiki.quantsoftware.org/index.php?title=Numpy_Tutorial_1
 
  
 
==Lesson 4: Statistical analysis of time series==
 
==Lesson 4: Statistical analysis of time series==

Revision as of 13:57, 6 April 2015

Lesson 1: Reading, slicing and plotting stock data

  • Overview of the data we'll be working with (from Yahoo!)
  • Introduction to our primary library: Pandas
  • Reading CSV data into Pandas
  • Filtering to specific dates
  • Plotting

Reading: "Python for Finance", Chapter 6: Financial time series

Lesson 2: Working with many stocks at once

  • Our target data frame structure
  • Address reverse order issue
  • Reading data for multiple stocks into the structure
  • Date slicing
  • Symbol slicing
  • Plotting
  • Normalizing

Lesson 3: The power of Numpy

  • What is Numpy and how it relates to Pandas
  • Why is Numpy powerful/important?
  • Creating Numpy arrays
  • Indexing and slicing Numpy arrays
  • Important data processing on Numpy arrays
  • Example use with pandas too

Reading: "Python for Finance", Chapter 4: Data types and structures

Lesson 4: Statistical analysis of time series

  • Gross statistics on dataframes
  • Rolling statistics on dataframes
  • Plotting a technical indicator (Bollinger Bands)
  • Memoizing

script

  • Rolling statistics example
    • Read SPY
    • 20 day rolling average
    • +- 20 day rolling stdev * 2
    • Plot above as Bollinger bands
  • Discussion of daily returns, what they are, how to calculate
  • [quiz: compute and plot SPY and XOM daily returns]
  • Show time series plot of daily rets
  • Show histogram of daily rets
  • Scatter plot (plot SPY vs XOM)
  • Compare SPY vs GLD. How can we quantify these differences?
  • Fit a line and plot it, print slope and corrcoef for SPY & XOM
  • Discussion of correlation not the same as slope

Lesson 5: Incomplete data

  • How incomplete data arises in financial data
  • Different approaches to dealing with it

script

  • [for this lesson: need to create 4 assets: SPY no missing, X ends midway, Y begins midway, Z has periodic outages]
  • Read SPY
    • 20 day rolling average
    • Plot
  • Attempt above with X, what happens? (incomplete data)
  • Look at data; NaN!
  • What to do?
  • Discussion & drawing of the types of incomplete data characterized by 4 examples above.
  • What is the proper way to handle?
  • [quiz: implement and plot fill forward on X]
  • Show Pandas methods for forward fill and backward fill, plot rolling averages

Lesson 6: Histograms and scatter plots

  • Histogram of daily returns
  • Compare SPY with XOM
  • Scatter plots
  • Correlation is not slope!
  • Compare SPY vs XOM, with SPY vs GLD scatter plots

Lesson 7: Computing statistics on a portfolio

  • Average daily return
  • Volatility: stddev of daily return (don't count first day)
  • Cumulative return
  • Relationship between cumulative and daily

script

  • Statistics we'll look at:
    • Average daily return
    • Stddev of daily return: volatility
    • Total return
    • Sharpe ratio
  • Buy and hold: SPY, GLD, GOOG, XOM, $100K each
  • Show how to read assets, normalize so each starts at $100K, goes forward.
  • [quiz: compute total daily portfolio value and daily rets for portfolio]
  • Show how to compute, avg daily rets, stdev, total ret, Sharpe ratio

Lesson 7: Optimizers: Building a parameterized model

  • Problem statement for an optimizer (inputs, outputs, assumptions)
  • How to build a parameterized model from real data using an optimizer

script

  • What does an optimizer do?
  • Show syntax of optimizer use
  • Create fake data
  • Try to fit it using an optimizer
  • [quiz: add another type of curve to fit (e.g., sine)]

Lesson 8: Optimizers: How to optimize a portfolio

  • Framing the portfolio problem for an optimizer
  • Constraints for an optimizer
  • Optimizing a portfolio

script

  • Frame the portfolio optimization problem for the optimizer
  • Add target return
  • Plug the parts together in code for 4 assets
  • [quiz: Add constraints on holdings]