Saturday, October 31, 2015

CLASS SIZE COMPLEXITY FOR ANY GIVEN COURSE

Class Size Complexity as a Measure of the Number of Relationships in the Classroom

This diagram represents network complexity for members of a particular class. (Click on the diagram for a larger view.) D-Comp represents Dyadic Complexity and is a measure of the number of one-to-one relationships within a class. It is determined by the formula {[CS x (CS-1)]/2} where CS represents class size. 3P-Comp represents Third Party Complexity and represents the number of third-party on-lookers that populate a group in which dialogue occurs within dyadic relationships. It is determined by the formula [CS x (CS-2)]. OC-Comp represents Overall Class Complexity and indicates the total network complexity for a class of a given size. As such it is a measure of the number of relations that must be managed by the structure and organization of a given course. It is determined by the formula {[CS x (CS-1)]/2} x (CS-2).

OC-Comp is the most important metric here, since it indicates the overall social pressure that group size plays on the total sum of individuals within the course. Note how fast it rises in relation to class size. A class of 12 members has an OC-Comp of 660. Add just one more student and the number increases to 858. By the time class size reaches 15, the OC-Comp is 1,365, more than double what it is for a class size of 12. At 18 members, OC-Comp is 2,448, at 25 it is 6,900, and at 40 it is 29,640.

One component of pedagogical strategy involves controlling OC-Comp. When properly harnessed social pressure can be used to motivate and invigorate course participation and student motivation. So, we will not be arguing to eliminate it. If it is not properly managed, then communication can break down. In this study, we will be searching for optimality of class size and organization to see if we can discover clues for successfully designing courses to manage social pressure using agent-based computer models.

All factors together suggest that for a rich, discussion-based, seminar-style course, twelve students is the magic number, eleven if you count the instructor/facilitator.

PAST PROJECTS FROM THE CSML

Below please find a sampling of some of the past projects done recently in the CSML that are unrelated to football:

  • Justin Simerly, Traffic COP (Controller Optimizing Productivity), an agent-based simulation of intelligent stop lights to optimize traffic flow. Winner of the University of Evansville's Most Outstanding Senior Project in Computer Science Award, 2014.
  • Austin Willis, Technological Enhancement of Humanistic Education: Modeling Student Interaction in a Classroom Setting. Supported by the UExplore Undergraduate Research Program, The University of Evansville. 2013.
  • Christopher Harrison, Transforming Standard Databases into Teleodynamic and Predictive Mechanisms. Supported by the UExplore Undergraduate Research Program, The University of Evansville. Some results of this study published as Information-Theoretic Teleodynamics in Natural and Artificial Systems. In A Computable Universe: Understanding Computation & Exploring Nature as Computation edited by Hector Zenil (World Scientific, 2012), forthcoming. Some results presented as 1) A Brief Introduction to Information-Theoretic Teleodynamics, Info-Metrics: Information Processing across the Sciences. Info-Metrics Institute Workshop on the Philosophy of Information, American University, Washington, D.C., October 3rd, 2011, and 2) Hybrid Networks: Transforming Networks for Social and Textual Analysis into Teleodynamic and Predictive Mechanisms. Institute for Advanced Topics in the Digital Humanities, sponsored by the National Endowment for the Humanities and the Institute for Pure and Applied Mathematics (IPAM), University of California, Los Angeles, October 20th-22nd, 2011.
  • Mason Blankenship and Justin Simerly, Noesis: Philosophical Research Online and the Indiana Philosophy Ontology Project. Supported by the National Endowment for the Humanities. 2011.
  • Derek Burrows, Dynamic Associative Network Automatic Document Classification. Supported by the National Endowment for the Humanities. Accepted for Presentation at the Undergraduate Conference in Math, Engineering and Science, University of Evansville, Evansville, Indiana, March 26th, 2011. Winner of the University of Evansville's Outstanding Senior Project in Computer Science Award, 2011.
  • Guy Wyant, Dynamic Associative Networks and Automatic Document Classification. Supported by the National Endowment for the Humanities. Winner of the University of Evansville's Outstanding Senior Project in Computer Science Award, 2010.
  • Michael Zlatkovsky, Modeling and Visualizing Dynamic Associative Networks: Towards Developing a More Robust and Biologically-Plausible Cognitive Model. See the companion website for details.Presented at the Undergraduate Conference in Math, Engineering and Science, University of Evansville, Evansville, Indiana, April 4th, 2009.Winner of the University of Evansville's Outstanding Senior Project in Computer Science Award, 2009.
  • Jonathan Clucas, Cognitive Transfer Via Analogical Reasoning: Applying Lessons to the Wason Selection Task. Presented at the National Conference on Undergraduate Research, Salisbury University, Salisbury, Maryland, April 10th-12th, 2008.

Thursday, October 29, 2015

WEEK EIGHT NFL PREDICTIONS

Below please find the 2015 week eight NFL predictions based on the model we built last year. This model uses a learning algorithm fed by data from the last seven weeks. To date, success rates this season are as follows: Week 2, 50%: Week 3, 62.5%: Week 4, 60%: Week 5, 64%; Week 6, 57%; Week 7, 50%.

We have slightly altered the 2014 model in various ways to yield three sets of predictions. This week we are taking the average of all three to make predictions. Model 1 (M1) is the precise procedure from last year and is based on normal averaging. So far, this has been the most accurate model. M2 is based on moving averages, and M3 is based on weighted percentages (thereby allowing games later in the season to count more than earlier games.)

Note that this week the model agreed with the crowd in 12 of the 14 games as opposed to 9 of the 14 from last week. The two games of contention are the SD @ BAL and NYG @ NO games, both of which the model predicts to be very close.

So here goes - (The % indicated after each game represents the crowd-sourced prediction from nfl.com as of this afternoon. The two digit decimal numbers following team names should be interpreted relative to each other for comparison to determine, in part, how close the game will be.):

  • MIA @ NE
    • NE (.63) over MIA (.55) - Easy win for NE
    • NE 36 over MIA 7 - Prediction Correct
    • Crowd Correct @ 89%
  • DET @ KC
    • KC (.51) over DET (.43) - Close game
    • KC 45 over DET 10 - Prediction Correct
    • Crowd Correct @ 71%
  • TB @ ATL
    • ATL (.57) over TB (.45) - Easy win for ATL
    • TB 23 over ATL 20 - Prediction Incorrect by 4 Points
    • Crowd Incorrect @ 96%
  • ARI @ CLE
    • ARI (.60) over CLE (.41) - Easy win for ARI
    • ARI 34 over CLE 20 - Prediction correct
    • Crowd Correct @ 96%
  • SF @ STL
    • STL (.51) over SF (.35) - Easy win for STL
    • STL 27 over SF 6 - Prediction Correct
    • Crowd Correct @ 90%
  • NYG @ NO
    • NYG (.52) over NO (.51) - Very close game
    • NO 52 over NYG 49 - Prediction Incorrect  by 4 Points
    • Crowd Incorrect @ 59%
  • MIN @ CHI
    • MIN (.56) over CHI (.39) - Easy win for MIN
    • MIN 23 over CHI 20 - Prediction Correct
    • Crowd Correct @ 82%
  • SD @ BAL
    • BAL (.47) over SD (.46) - Very close game
    • BAL 29 over SD 26 - Prediction Correct
    • Crowd Incorrect @ 59%
  • CIN @ PIT
    • CIN (.59) over PIT (.54) - Close game
    • CIN 16 over PIT 10 - Prediction Correct
    • Crowd Correct @ 68%
  • TEN @ HOU
    • HOU (.47) over TEN (.41) - Close game
    • HOU 20 over TEN 6 - Prediction Correct
    • Crowd Correct @ 71%
  • NYJ @ OAK
    • NYJ (.56) over OAK (.48) - Easy win for NYJ
    • OAK 34 over NYJ 20 - Prediction Incorrect
    • Crowd Incorrect @ 72%
  • SEA @ DAL
    • SEA (.66) over DAL (.41) - Easy win for SEA
    • SEA 13 over DAL 12 - Prediction Correct
    • Crowd Correct @ 84%
  • GB @ DEN
    • GB (.65) over DEN (.57) - Interesting game
    • DEN 29 over GB 10 - Prediction Incorrect
    • Crowd Incorrect @ 75%
  • IND @ CAR
    • CAR (.60) over IND (.48) - Easy win for CAR
    • CAR 29 over IND 26 - Prediction Correct
    • Crowd Correct @ 88%

Games to watch: NYG @ NO, SD @ BAL, and GB @ DEN.

WEEK SEVEN NFL RESULTS

Below please find the 2015 week seven NFL results based on the model we built last year. This model uses a learning algorithm fed by data from the last six weeks. To date, success rates this season are as follows: Week 2 - 50%, Week 3 - 62.5%, Week 4 - 60%, Week 5 - 64%, Week 6 - 57% and then this week - 50%.

I did say earlier that wasn't feel very good about the reliability of these predictions, largely due to larger than usual divergence from the crowd's predictions. The crowd faired better this week than the model in getting 10 correct of the 14 for a 71% accuracy rating. No doubt, the crude model from last year has its limitations. Even so, it seems to have done better last year.

Progress continues on our drive analysis network, though preliminary testing on components of the network thus far look promising, it would be premature to say what the final reliability of drive analysis will be. The more I watch these games, especially this year, the more I am convinced that depth charts matter, since injuries play such a major role in game outcomes. If so, we can account for them by adding additional layers to the network in future semesters.

  • SEA @ SF - Easy win for SEA
    • M1 - SEA (.56) over SF (.41)
    • M2 - SEA (.51) over SF (.46)
    • M3 - SEA (.55) over SF (.39)
      • Overall Prediction: SEA over SF
      • SEA 20 over SF 3 - Prediction Correct
      • Crowd Correct @ 81%
  • BUF @ JAX - ???
    • M1 - BUF (.51) over JAX (.40)
    • M2 - BUF (.44) over JAX (.41)
    • M3 - BUF (.47) over JAX (.40)
      • Overall Prediction: BUF over JAX
      • JAX 34 over BUF 31 - Prediction Incorrect
      • Crowd Incorrect @ 88%
  • MIN @ DET - Close game
    • M1 DET (.58) over MIN (.51)
    • M2 DET (.56) over MIN (.56)
    • M3 MIN (.59) over DET (.58)
      • Overall Prediction: DET over MIN
      • MIN 28 over DET 19 - Prediction Incorrect
      • Crowd Correct @ 67%
  • NO @ IND - Close game
    • M1 - NO (.45) over IND (.45)
    • M2 - NO (.50) over IND (.48)
    • M3 - IND (.48) over NO (.47)
      • Overall Prediction: NO over IND
      • NO 27 over IND 21 - Prediction Correct
      • Crowd Incorrect @ 79%
  • PIT @ KC - Easy win for PIT
    • M1 - PIT (.58) over KC (.45)
    • M2 - PIT (.60) over KC (.42)
    • M3 - PIT (.59) over KC (.42)
      • Overall Prediction: PIT over KC
      • KC 23 over PIT 13 - Prediction Incorrect
      • Crowd Incorrect @ 86%
  • HOU @ MIA - Close game
    • M1 - MIA (.49) over HOU (.48)
    • M2 - MIA (.58) over HOU (.52)
    • M3 - MIA (.53) over HOU (.49)
      • Overall Prediction: MIA over HOU
      • MIA 44 over HOU 26 - Prediction Correct
      • Crowd Correct @ 73%
  • NYJ @ NE - Close game
    • M1 - NE (.65) over NYJ (.64)
    • M2 - NE (.65) over NYJ (.62)
    • M3 - NE (.70) over NYJ (.65)
      • Overall Prediction: NE over NYJ
      • NE 30 over NYJ 23 - Prediction Correct
      • Crowd Correct @ 85%
  • CLE @ STL - Close game
    • M1 - CLE (.47) over STL (.39)
    • M2 - CLE (.48) over STL (.37)
    • M3 - CLE (.48) over STL (.39)
      • Overall Prediction: CLE over STL
      • STL 24 over CLE 6 - Prediction Incorrect
      • Crowd Correct @ 78%
  • ATL @ TEN - Easy will for ATL
    • M1 - ATL (.55) over TEN (.45)
    • M2 - ATL (.50) over TEN (.35)
    • M3 - ATL (.54) over TEN (.41)
      • Overall Prediction: ATL over TEN
      • ATL 10 over TEN 7 - Prediction Correct
      • Crowd Correct @ 97%
  • TB @ WAS - Close game
    • M1 - WAS (.47) over TB (.42)
    • M2 - TB (.46) over WAS (.42)
    • M3 - TB (.46) over WAS (.45)
      • Overall Prediction: TB over WAS
      • WAS 31 over TB 30 - Prediction Incorrect
      • Crowd Correct @ 67%
  • OAK @ SD - Close game
    • M1 - SD (.45) over OAK (.45)
    • M2 - SD (.44) over OAK (.43)
    • M3 - OAK (.48) over SD (.44)
      • Overall Prediction: SD over OAK
      • OAK 37 over SD 29 - Prediction Incorrect
      • Crowd Incorrect @ 81%
  • DAL @ NYG - Close game
    • M1 - NYG (.50) over DAL (.45)
    • M2 - NYG (.39) over DAL (.33)
    • M3 - NYG (.41) over DAL (.47)
      • Overall Prediction: NYG over DAL
      • NYG 27 over DAL 20 - Prediction Correct
      • Crowd Correct @ 74%
  • PHI @ CAR - Close game
    • M1 - CAR (.60) over PHI (.56)
    • M2 - PHI (.69) over CAR (.57)
    • M3 - PHI (.62) over CAR (.61)
      • Overall Prediction: PHI over CAR
      • CAR 27 over PHI 16 - Prediction Incorrect
      • Crowd Correct @ 82%
  • BAL @ ARI - ???
    • M1 - ARI (.62) over BAL (.46)
    • M2 - ARI (.50) over BAL (.46)
    • M3 - ARI (.57) over BAL (.47)
      • Overall Prediction: ARI over BAL
      • ARI 26 over BAL 18 - Prediction Correct
      • Crowd Correct @ 95%

Saturday, October 24, 2015

CALL FOR SPRING 2016 INTERNS

Three internship spots for working in the Cognitive Science Modeling Laboratory (CSML) are available for this coming Spring. Opportunities are open to University of Evansville students from all four years (freshman through senior year) and any major, but are also limited to those who’ve had some computer programming or scripting experience. Excel VBA or C# preferred, but not necessary. Additional background in American professional football, statistics, econometrics, numerical methods, network analysis, machine learning, or any other predictive modeling or forecasting techniques are a plus. If you are interested, please complete an application and email it to Dr. Beavers at tb2@evansville.edu by 5 pm on November 9th. Interviews for selected applicants will follow after.

Application forms are available at: http://tinyurl.com/k95nb3f.

To learn more about the CSML, our current project, and our internships please see:

Thursday, October 22, 2015

WEEK SEVEN NFL PREDICTIONS

Below please find the 2015 week seven NFL predictions based on the model we built last year. This model uses a learning algorithm fed by data from the last six weeks. To date, success rates this season are as follows: Week 2, 50%: Week 3, 62.5%: Week 4, 60%: Week 5, 64%; Week 6, 57%.

We have slightly altered the 2014 model in various ways to yield three sets of predictions. Model 1 (M1) is the procedure from last year and is based on normal averaging. So far, this has been the most accurate model. M2 is based on moving averages, and M3 is based on weighted percentages (thereby allowing games later in the season to count more than earlier games.)

Note that the model agreed with the crowd in only 9 of the 14 games as opposed to 12 of the 14 from last week.

Bottom line - I'm not feeling very confident in the predictions below. However, our new model is developing well. Click here to learn more.

So here goes - (The % indicated after each game represents the crowd-sourced prediction from nfl.com as of this afternoon. The two digit decimal numbers following team names should be interpreted relative to each other for comparison of the models and to determine, in part, how close the game will be.):

  • SEA @ SF - Easy win for SEA
    • M1 - SEA (.56) over SF (.41)
    • M2 - SEA (.51) over SF (.46)
    • M3 - SEA (.55) over SF (.39)
      • Overall Prediction: SEA over SF
      • SEA 20 over SF 3 - Prediction Correct
      • Crowd Correct @ 81%
  • BUF @ JAX - ???
    • M1 - BUF (.51) over JAX (.40)
    • M2 - BUF (.44) over JAX (.41)
    • M3 - BUF (.47) over JAX (.40)
      • Overall Prediction: BUF over JAX
      • JAX 34 over BUF 31 - Prediction Incorrect
      • Crowd Incorrect @ 88%
  • MIN @ DET - Close game
    • M1 DET (.58) over MIN (.51)
    • M2 DET (.56) over MIN (.56)
    • M3 MIN (.59) over DET (.58)
      • Overall Prediction: DET over MIN
      • MIN 28 over DET 19 - Prediction Incorrect
      • Crowd Correct @ 67%
  • NO @ IND - Close game
    • M1 - NO (.45) over IND (.45)
    • M2 - NO (.50) over IND (.48)
    • M3 - IND (.48) over NO (.47)
      • Overall Prediction: NO over IND
      • NO 27 over IND 21 - Prediction Correct
      • Crowd Incorrect @ 79%
  • PIT @ KC - Easy win for PIT
    • M1 - PIT (.58) over KC (.45)
    • M2 - PIT (.60) over KC (.42)
    • M3 - PIT (.59) over KC (.42)
      • Overall Prediction: PIT over KC
      • KC 23 over PIT 13 - Prediction Incorrect
      • Crowd Incorrect @ 86%
  • HOU @ MIA - Close game
    • M1 - MIA (.49) over HOU (.48)
    • M2 - MIA (.58) over HOU (.52)
    • M3 - MIA (.53) over HOU (.49)
      • Overall Prediction: MIA over HOU
      • MIA 44 over HOU 26 - Prediction Correct
      • Crowd Correct @ 73%
  • NYJ @ NE - Close game
    • M1 - NE (.65) over NYJ (.64)
    • M2 - NE (.65) over NYJ (.62)
    • M3 - NE (.70) over NYJ (.65)
      • Overall Prediction: NE over NYJ
      • NE 30 over NYJ 23 - Prediction Correct
      • Crowd Correct @ 85%
  • CLE @ STL - Close game
    • M1 - CLE (.47) over STL (.39)
    • M2 - CLE (.48) over STL (.37)
    • M3 - CLE (.48) over STL (.39)
      • Overall Prediction: CLE over STL
      • STL 24 over CLE 6 - Prediction Incorrect
      • Crowd Correct @ 78%
  • ATL @ TEN - Easy will for ATL
    • M1 - ATL (.55) over TEN (.45)
    • M2 - ATL (.50) over TEN (.35)
    • M3 - ATL (.54) over TEN (.41)
      • Overall Prediction: ATL over TEN
      • ATL 10 over TEN 7 - Prediction Correct
      • Crowd Correct @ 97%
  • TB @ WAS - Close game
    • M1 - WAS (.47) over TB (.42)
    • M2 - TB (.46) over WAS (.42)
    • M3 - TB (.46) over WAS (.45)
      • Overall Prediction: TB over WAS
      • WAS 31 over TB 30 - Prediction Incorrect
      • Crowd Correct @ 67%
  • OAK @ SD - Close game
    • M1 - SD (.45) over OAK (.45)
    • M2 - SD (.44) over OAK (.43)
    • M3 - OAK (.48) over SD (.44)
      • Overall Prediction: SD over OAK
      • OAK 37 over SD 29 - Prediction Incorrect
      • Crowd Incorrect @ 81%
  • DAL @ NYG - Close game
    • M1 - NYG (.50) over DAL (.45)
    • M2 - NYG (.39) over DAL (.33)
    • M3 - NYG (.41) over DAL (.47)
      • Overall Prediction: NYG over DAL
      • NYG 27 over DAL 20 - Prediction Correct
      • Crowd Correct @ 74%
  • PHI @ CAR - Close game
    • M1 - CAR (.60) over PHI (.56)
    • M2 - PHI (.69) over CAR (.57)
    • M3 - PHI (.62) over CAR (.61)
      • Overall Prediction: PHI over CAR
      • CAR 27 over PHI 16 - Prediction Incorrect
      • Crowd Correct @ 82%
  • BAL @ ARI - ???
    • M1 - ARI (.62) over BAL (.46)
    • M2 - ARI (.50) over BAL (.46)
    • M3 - ARI (.57) over BAL (.47)
      • Overall Prediction: ARI over BAL
      • ARI 26 over BAL 18 - Prediction Correct
      • Crowd Correct @ 95%

Games to watch: NO @ IND, NYJ @ NE, CLE @ STL (Yeah, that's right. The Cleveland game), PHI @ CAR.

Wednesday, October 21, 2015

2015-2016 CSML "PREDICTING THE NFL" UPDATE

Attempted Pattern Detection of One Style of Play

In an earlier post, A Glimpse of Things to Come, I announced the general direction of our project for this academic year. Things are progressing well, but a brief update at this point is in order.

The general approach that we are taking to improve prediction over last year's model is by way of an AI network analysis model of various play strategies.

Part of this initiative is the creation of a multi-layered feature detection network that implicitly compared the various drive profiles of possible play over the last several years.

Currently, we are compiling data from all games since 2000 to create a network base of more than 3500 games that will serve as our training set. At the highest level of analysis the drive profile is no longer attached to various teams. Rather, we are inspecting which styles of play can beat other styles of play regardless of team.

The preliminary feature detection network provided a coarse-grained analysis. However, as many a network specialist would suspect, many games share many of the same features. Thus, finer grained analysis means that we must control for information overflow due to excessive commonalities. To do this, we are adding two layers to the detection network, one to account for information entropy and the other to show only those nodes that pass a dynamic threshold. The result will be a network that isolates which drive features are most pertinent to the final results of a game ... hopefully.

The feature detection layer will be five-layered, including both its input and output layer. On top of that, we are building several decision layers to match particular play styles to particular game outcomes. This will include matching of point spreads, which we hope to be able to get the network to determine for specific games in time.

After the network is complete, we will use the current season's play styles for each team to make predictions about the final result of each game. The early results are very preliminary, but the network is already better at guessing game winners based solely on two weeks of data from the current season than the method from last year.

Whether this approach will work as well as we would like remains to be seen. Minimally, however, the network will allow for a rigorous comparison of teams based on similarity measures. Thus, for any target team, we will be able to provide an ordered list of the 31 remaining teams ranked by their similarities to the target. This will allow us to predict that if team X beats Y and if team Z has a drive profile very similar to team X, it too should be able to beat team Y.

If this does not work by itself, we still have other components to add to the network to (try to) improve results. Our target is to hit in the 75%-80% range with regularity.

CSML POWER RANKINGS AS OF 10/21/15

Below are the CSML power rankings as the current date. They were generated by adding the current values for M1, M2 and M3 for each team as of today. 

Model 1 (M1) is the procedure from last year and is based on normal averaging. So far, this has been the most accurate model. M2 is based on moving averages, and M3 is based on weighted percentages (thereby allowing games later in the season to count more than earlier games.)


NE 2.01
GB 1.95
NYJ 1.91
PHI 1.87
CIN 1.78
CAR 1.78
PIT 1.77
DEN 1.72
ARI 1.69
MIN 1.66
SEA 1.62
MIA 1.61
ATL 1.59
HOU 1.48
CLE 1.43
NO 1.43
BUF 1.41
IND 1.41
BAL 1.40
NYG 1.37
OAK 1.36
TB 1.34
SD 1.34
WAS 1.34
DET 1.31
KC 1.29
SF 1.26
TEN 1.22
JAX 1.21
DAL 1.18
CHI 1.17
STL 1.15

All in all, they look pretty intuitive to me, though prior to the season I would have guessed that SEA, IND and DAL would be higher.