Defeat learners: - Revision history

Tucker: Replaced content with "Go here defeat_learners"

2017-09-25T10:15:52Z

Replaced content with "Go here defeat_learners"

Tucker at 10:13, 25 September 2017

2017-09-25T10:13:28Z

Tucker: Created page with "==Draft== Note: add author() method ==Updates / FAQs== * '''March 15, 2017''' API revised to include random number seed. * '''March 20, 2017''' Homework description finalize..."

2017-08-27T20:25:14Z

Created page with "==Draft== Note: add author() method ==Updates / FAQs== * '''March 15, 2017''' API revised to include random number seed. * '''March 20, 2017''' Homework description finalize..."

New page

==Draft==
Note: add author() method

==Updates / FAQs==

* '''March 15, 2017''' API revised to include random number seed.
* '''March 20, 2017''' Homework description finalized and released..

==Overview==

For this homework you will generate data that you believe will work better for one learner than another. This will test your understanding of the strengths and weaknesses of various learners. The two learners you should aim your datasets at are:
* A random tree learner with leaf_size = 1.
* The LinRegLearner provided as part of the repo.

Your data generation should use a random number generator as part of its data generation process. We will pass your generators a random number seed. Whenever the seed is the same you should return exactly the same data set. Different seeds should result in different data sets.

==Template and Data==

If necessary, update your version of the repo following the instructions here: [[http://quantsoftware.gatech.edu/ML4T_Software_Setup#Updating_the_repository]]. You will see the following files in your directory mc3_h1:

* <tt>gen_data.py</tt> An implementation of the code you are supposed to provide: It includes two functions that each return a data set. Note that the data sets those functions return DO NOT satisfy the requirements for the homework. But they do show you how you can generate a data set.
* <tt>LinRegLearner.py</tt> Our friendly, working, correct, linear regression learner. It is used by the testing code.
* <tt>RTLearner.py</tt> A working, but INCORRECT, Random Tree learner. Replace it with your working, correct RTLearner.
* <tt>testbest4.py</tt> Code that calls the two data set generating functions and tests them against the two learners.

==Generate your own datasets==

Create a Python program called gen_data.py that implements two functions. The two functions should be named as follows, and support the following API:

X1, Y1 = best4LinReg(seed = 5)
X2, Y2 = best4RT(seed = 5)

* '''seed''' Your data generation should use a random number generator as part of its data generation process. We will pass your generators a random number seed. Whenever the seed is the same you should return exactly the same data set. Different seeds should result in different data sets.

best4LinReg() should return data that performs significantly better (see rubric) with LinRegLearner than RTLearner. best4RT() should return data that performs significantly better with RTLearner than LinRegLearner.

Each data set should include from 2 to 1000 columns in X, and one column in Y. The data should contain from 10 (minimum) to 1000 (maximum) rows.

==What to turn in==
Be sure to follow these instructions diligently!

Via T-Square, submit as attachment (no zip files; refer to schedule for deadline):

* Your code as <tt>gen_data.py</tt>

Unlimited resubmissions are allowed up to the deadline for the project.

==Rubric==

Deductions:
* Does either dataset returned contain fewer or more than the allowed number of samples? -20% each.
* Does either dataset returned contain fewer or more than the allowed number of dimensions in X? -20% each.
* When the seed is the same does the best4LinReg dataset generator return the same data? -20% otherwise.
* When the seed is the same does the best4RT dataset generator return the same data? -20% otherwise.
* When the seed is different does the best4LinReg dataset generator return different data? -20% otherwise.
* When the seed is different does the best4RT dataset generator return different data? -20% otherwise.

For best4LinReg:
* We will run 15 test cases and select the best 10. For each successful test +5 points (total of 50%)
* For each test case we will randomly select 60% of the data for training and 40% for testing.
* Success for each case is defined as: RMSE LinReg < RMSE RT * 0.9

For best4RT:
* We will run 15 test cases and select the best 10. For each successful test +5 points (total of 50%)
* For each test case we will randomly select 60% of the data for training and 40% for testing.
* Success for each case is defined as: RMSE RT < RMSE LinReg * 0.9

==Required, Allowed & Prohibited==

Required:
* No reading of data from files.
* Your project must be coded in Python 2.7.x.
* Your code must run on one of the university-provided computers (e.g. buffet02.cc.gatech.edu), or on one of the provided virtual images.
* Your code must run in less than 5 seconds on one of the university-provided computers.
* The code you submit should NOT include any data reading routines. You should generate all of your data within your functions.
* The code you submit should NOT generate any output: No prints, no charts, etc.

Allowed:
* You can develop your code on your personal machine, but it must also run successfully on one of the university provided machines or virtual images.
* Your code may use standard Python libraries.
* You may use the NumPy, SciPy, matplotlib and Pandas libraries. Be sure you are using the correct versions.
* You may reuse sections of code (up to 5 lines) that you collected from other students or the internet.
* Code provided by the instructor, or allowed by the instructor to be shared.
* Cheese.

Prohibited:
* Any reading of data files.
* Any libraries not listed in the "allowed" section above.
* Any code you did not write yourself (except for the 5 line rule in the "allowed" section).
* Any Classes (other than Random) that create their own instance variables for later use (e.g., learners like kdtree).
* Code that includes any data reading routines. The provided testlearner.py code reads data for you.
* Code that generates any output when verbose = False: No prints, no charts, etc.
* Ducks and wood.

==Legacy==

[[MC3-Homework-1-legacy]]

← Older revision		Revision as of 10:15, 25 September 2017
Line 1:		Line 1:
	Go here [[defeat_learners]]		Go here [[defeat_learners]]
−
−	~~==Draft==~~
−	~~Note: add author() method~~
−
−	~~==Updates / FAQs==~~
−
−	* '''March 15, 2017''' API revised to include random number seed.
−	* '''March 20, 2017''' Homework description finalized and released..
−
−	~~==Overview==~~
−
−	For this homework you will generate data that you believe will work better for one learner than another. This will test your understanding of the strengths and weaknesses of various learners. The two learners you should aim your datasets at are:
−	* A random tree learner with leaf_size = 1.
−	* The LinRegLearner provided as part of the repo.
−
−	Your data generation should use a random number generator as part of its data generation process. We will pass your generators a random number seed. Whenever the seed is the same you should return exactly the same data set. Different seeds should result in different data sets.
−
−	~~==Template and Data==~~
−
−	If necessary, update your version of the repo following the instructions here: [[http://quantsoftware.gatech.edu/ML4T_Software_Setup#Updating_the_repository]]. You will see the following files in your directory mc3_h1:
−
−	* <tt>gen_data.py</tt> An implementation of the code you are supposed to provide: It includes two functions that each return a data set. Note that the data sets those functions return DO NOT satisfy the requirements for the homework. But they do show you how you can generate a data set.
−	* <tt>LinRegLearner.py</tt> Our friendly, working, correct, linear regression learner. It is used by the testing code.
−	* <tt>RTLearner.py</tt> A working, but INCORRECT, Random Tree learner. Replace it with your working, correct RTLearner.
−	* <tt>testbest4.py</tt> Code that calls the two data set generating functions and tests them against the two learners.
−
−	~~==Generate your own datasets==~~
−
−	~~Create a Python program called gen_data.py that implements two functions. The two functions should be named as follows, and support the following API:~~
−
−	~~X1, Y1 = best4LinReg(seed = 5)~~
−	~~X2, Y2 = best4RT(seed = 5)~~
−
−	* '''seed''' Your data generation should use a random number generator as part of its data generation process. We will pass your generators a random number seed. Whenever the seed is the same you should return exactly the same data set. Different seeds should result in different data sets.
−
−	best4LinReg() should return data that performs significantly better (see rubric) with LinRegLearner than RTLearner. best4RT() should return data that performs significantly better with RTLearner than LinRegLearner.
−
−	~~Each data set should include from 2 to 1000 columns in X, and one column in Y. The data should contain from 10 (minimum) to 1000 (maximum) rows.~~
−
−	~~==What to turn in==~~
−	~~Be sure to follow these instructions diligently!~~
−
−	~~Via T-Square, submit as attachment (no zip files; refer to schedule for deadline):~~
−
−	* Your code as <tt>gen_data.py</tt>
−
−	~~Unlimited resubmissions are allowed up to the deadline for the project.~~
−
−	~~==Rubric==~~
−
−	~~Deductions:~~
−	* Does either dataset returned contain fewer or more than the allowed number of samples? -20% each.
−	* Does either dataset returned contain fewer or more than the allowed number of dimensions in X? -20% each.
−	* When the seed is the same does the best4LinReg dataset generator return the same data? -20% otherwise.
−	* When the seed is the same does the best4RT dataset generator return the same data? -20% otherwise.
−	* When the seed is different does the best4LinReg dataset generator return different data? -20% otherwise.
−	* When the seed is different does the best4RT dataset generator return different data? -20% otherwise.
−
−	~~For best4LinReg:~~
−	* We will run 15 test cases and select the best 10. For each successful test +5 points (total of 50%)
−	* For each test case we will randomly select 60% of the data for training and 40% for testing.
−	* Success for each case is defined as: RMSE LinReg < RMSE RT * 0.9
−
−	~~For best4RT:~~
−	* We will run 15 test cases and select the best 10. For each successful test +5 points (total of 50%)
−	* For each test case we will randomly select 60% of the data for training and 40% for testing.
−	* Success for each case is defined as: RMSE RT < RMSE LinReg * 0.9
−
−	~~==Required, Allowed & Prohibited==~~
−
−	~~Required:~~
−	* No reading of data from files.
−	* Your project must be coded in Python 2.7.x.
−	* Your code must run on one of the university-provided computers (e.g. buffet02.cc.gatech.edu), or on one of the provided virtual images.
−	* Your code must run in less than 5 seconds on one of the university-provided computers.
−	* The code you submit should NOT include any data reading routines. You should generate all of your data within your functions.
−	* The code you submit should NOT generate any output: No prints, no charts, etc.
−
−	~~Allowed:~~
−	* You can develop your code on your personal machine, but it must also run successfully on one of the university provided machines or virtual images.
−	* Your code may use standard Python libraries.
−	* You may use the NumPy, SciPy, matplotlib and Pandas libraries. Be sure you are using the correct versions.
−	* You may reuse sections of code (up to 5 lines) that you collected from other students or the internet.
−	* Code provided by the instructor, or allowed by the instructor to be shared.
−	* Cheese.
−
−	~~Prohibited:~~
−	* Any reading of data files.
−	* Any libraries not listed in the "allowed" section above.
−	* Any code you did not write yourself (except for the 5 line rule in the "allowed" section).
−	* Any Classes (other than Random) that create their own instance variables for later use (e.g., learners like kdtree).
−	* Code that includes any data reading routines. The provided testlearner.py code reads data for you.
−	* Code that generates any output when verbose = False: No prints, no charts, etc.
−	* Ducks and wood.
−
−	~~==Legacy==~~
−
−	~~[[MC3-Homework-1-legacy]]~~

← Older revision		Revision as of 10:13, 25 September 2017
Line 1:		Line 1:
		+	Go here [[defeat_learners]]
		+
	==Draft==		==Draft==
	Note: add author() method		Note: add author() method