RPI Manipulation - cougarguard.com — unofficial BYU Cougars / LDS sports, football, basketball forum and message board

jay santos · 03-07-2006, 08:38 PM

I've been intrigued by the high RPI's of the Missouri Valley Conference and wanted to test a hypothesis that some schools/conferences are getting an unfair advantage from the quirky RPI formula.

I do a statistical model of college football and basketball. My model usually is pretty close to Sagarin and usually fares in the top 10 of the models evaluated at http://www.masseyratings.com/cb/compare.htm.

I compared my model to RPI. I did this at a conference level. Out of the 32 conferences, I isolated ten conferences--five that did the worst in the RPI's compared to my model and the five that did the best in the RPI's compared to my model.

The five conferences that were the most RPI friendly (RPI+)

Big 10
Big South
Colonial
MAAC
MVC

Five most RPI non-friendly (RPI-)

Pac 10
MWC
CUSA
BW
MCon

The difference was pretty significant.

My model
RPI+ average rank #125
RPI- average rank #115

RPI
RPI+ average rank #113
RPI- average rank #141

I tested several theories. My model includes margin of victory--so I looked at that, but both groups had roughly equal MOV. SOS was pretty close. W-L was very close. I tested different SOS calculations, but none mattered much.

Then I found the difference maker: the home/away split. This past year, the RPI formula made a significant change in the formula by accounting for home and away wins and losses in a different way. The RPI makers had their hearts in the right places--it's a good idea to account for home/away, but they overcorrected.

The home/away split accounted for a huge amount of the variance between the two models.

RPI+ conferences
Home: 47%
Away: 44%
Neutral: 9%

RPI- conferences
Home: 54%
Away: 40%
Neutral: 6%

Then I took all 32 conferences and charted their power ratings with their home game %. The slopes were about equal so I charted them on top of each other. The results were pretty glaring. All five of my RPI+ points were above the power rating line and all five of my RPI- points were below the line. The rest of the conferences correlated with my RPI+- analysis.

So, in summary, the RPI really screwed up this past year by overcorrecting on the home/away piece of the formula and conferences such as the MVC and the Colonial are getting the benefit and conferences such as the MWC and C-USA are getting screwed.

Also of note, my model already accounts for home/away in both the MOV piece and the SOS piece.

The RPI is a bad formula. When you look at the computer rankings at http://www.masseyratings.com/cb/compare.htm, the RPI is one of the worst that is ranked. As much as we hate the BCS, the BCS formula is a much better formula than RPI. This would be like choosing one really bad computer model and calling that the entire BCS formula in football. Basketball needs to move to a BCS-like formula.

il Padrino Ute · 03-08-2006, 01:22 AM

I disagree that basketball needs to use a BCS-like formula only because the BCS is all about hording money for six conferences and a self-proclaimed national championship.

I do agree that the RPI is more flawed than what the BCS uses.

Still, the NCAA tournament, regardless what they use for seedings, is still the best thing in college sports - or any sport championship for that matter.

You do this in your spare time? That shames me. I use spare time to play X-box.

ops:

il Padrino Ute · 03-08-2006, 01:27 AM

I loved looking at your golf ranking. With your formula, it looks like it's no longer TIger and everyone else, but Tiger/Vijay and everyone else.

I appreciate the link to your page. I'll be keeping my eye on the baseball and golf rankings all season. Well done.

jay santos · 03-08-2006, 02:27 AM

Quote:

Originally Posted by il Padrino Ute

I loved looking at your golf ranking. With your formula, it looks like it's no longer TIger and everyone else, but Tiger/Vijay and everyone else.

I appreciate the link to your page. I'll be keeping my eye on the baseball and golf rankings all season. Well done.

No, that's not my web page. That was for reference to show where many of the computer models are listed. I only do college football and basketball.

I believe BCS style rankings should be used for determining the field of 64, not that basketball should drop the tourney and go with a BCS post season format.

MikeWaters · 03-08-2006, 02:54 AM

jay, is your model on the page? if so, which one is it?

jay santos · 03-08-2006, 03:29 AM

Quote:

Originally Posted by MikeWaters

jay, is your model on the page? if so, which one is it?

\

No, I've been doing it for about 10 years but I've never submitted it. Didn't want the burden of having to update it and send it in a timely fashion. Most of my effort has gone into trying to develop a system that can beat Vegas. After a lot of work I think it's not possible. Actually I do have a system that I think might work and I've had moderate success the last two years, but it's on a small subset of the total games and I still am not totally sure I have enough data to prove it. Plus, I think Vegas gets smarter every year.

03-08-2006, 01:22 AM	#2
il Padrino Ute Board Pinhead Join Date: Jan 2006 Location: In the basement of my house, Murray, Utah. Posts: 15,941	I disagree that basketball needs to use a BCS-like formula only because the BCS is all about hording money for six conferences and a self-proclaimed national championship. I do agree that the RPI is more flawed than what the BCS uses. Still, the NCAA tournament, regardless what they use for seedings, is still the best thing in college sports - or any sport championship for that matter. You do this in your spare time? That shames me. I use spare time to play X-box. ops: __________________ "The beauty of baseball is not having to explain it." - Chuck Shriver "This is now the joke that stupid people laugh at." - Christopher Hitchens on IQ jokes about GWB.

03-08-2006, 01:27 AM	#3
il Padrino Ute Board Pinhead Join Date: Jan 2006 Location: In the basement of my house, Murray, Utah. Posts: 15,941	I loved looking at your golf ranking. With your formula, it looks like it's no longer TIger and everyone else, but Tiger/Vijay and everyone else. I appreciate the link to your page. I'll be keeping my eye on the baseball and golf rankings all season. Well done. __________________ "The beauty of baseball is not having to explain it." - Chuck Shriver "This is now the joke that stupid people laugh at." - Christopher Hitchens on IQ jokes about GWB.

03-07-2006, 08:38 PM	#1
jay santos Senior Member Join Date: Jan 2006 Posts: 6,177	RPI Manipulation I've been intrigued by the high RPI's of the Missouri Valley Conference and wanted to test a hypothesis that some schools/conferences are getting an unfair advantage from the quirky RPI formula. I do a statistical model of college football and basketball. My model usually is pretty close to Sagarin and usually fares in the top 10 of the models evaluated at http://www.masseyratings.com/cb/compare.htm. I compared my model to RPI. I did this at a conference level. Out of the 32 conferences, I isolated ten conferences--five that did the worst in the RPI's compared to my model and the five that did the best in the RPI's compared to my model. The five conferences that were the most RPI friendly (RPI+) Big 10 Big South Colonial MAAC MVC Five most RPI non-friendly (RPI-) Pac 10 MWC CUSA BW MCon The difference was pretty significant. My model RPI+ average rank #125 RPI- average rank #115 RPI RPI+ average rank #113 RPI- average rank #141 I tested several theories. My model includes margin of victory--so I looked at that, but both groups had roughly equal MOV. SOS was pretty close. W-L was very close. I tested different SOS calculations, but none mattered much. Then I found the difference maker: the home/away split. This past year, the RPI formula made a significant change in the formula by accounting for home and away wins and losses in a different way. The RPI makers had their hearts in the right places--it's a good idea to account for home/away, but they overcorrected. The home/away split accounted for a huge amount of the variance between the two models. RPI+ conferences Home: 47% Away: 44% Neutral: 9% RPI- conferences Home: 54% Away: 40% Neutral: 6% Then I took all 32 conferences and charted their power ratings with their home game %. The slopes were about equal so I charted them on top of each other. The results were pretty glaring. All five of my RPI+ points were above the power rating line and all five of my RPI- points were below the line. The rest of the conferences correlated with my RPI+- analysis. So, in summary, the RPI really screwed up this past year by overcorrecting on the home/away piece of the formula and conferences such as the MVC and the Colonial are getting the benefit and conferences such as the MWC and C-USA are getting screwed. Also of note, my model already accounts for home/away in both the MOV piece and the SOS piece. The RPI is a bad formula. When you look at the computer rankings at http://www.masseyratings.com/cb/compare.htm, the RPI is one of the worst that is ranked. As much as we hate the BCS, the BCS formula is a much better formula than RPI. This would be like choosing one really bad computer model and calling that the entire BCS formula in football. Basketball needs to move to a BCS-like formula.

03-08-2006, 02:54 AM	#5
MikeWaters Demiurge Join Date: Aug 2005 Posts: 36,365	jay, is your model on the page? if so, which one is it?