A Logical Fallacy From Sabermetrics

[spark240]

Originally Posted by ipitch

I'm saying that they perform below their average in less important situations.

If they perform above their average in some situations, they have to perform below their average in others. If they can match that variance to the importance of the situation, rather than the inverse, that would be very desirable indeed.

[ipitch]

Originally Posted by spark240

If they perform above their average in some situations, they have to perform below their average in others.

I know. But, I don't think some people realize this. In order to be clutch, you have to be bad in some other situation.

[SABR Matt]

No...in order to be clutch as rigorously defined by the typical (falsely pursued) sabermetic experiment, you have to be above your own average in the clutch and below in other situations...but the right answer is as Tom Tango emphasized in his book...the best hitters in clutch situations will be...the best overall hitters. In other words...if you're a good hitter, you'll still be a good hitter when the game is on the line. I don't think there exists a player who is significantly worse than his averages in clutch situations to the point where his good surface stats are biased by performances in low leverage situations.

Not because players don't feel pressure...but because they respond to pressure in similar enough ways that the differences tend to be minor. That doesn't mean some don't suffer or do better than the norm...but it's going to be hard to see it in the metrics.

[NJYankeeFan]

Originally Posted by Pete Rose Rounding Third

I assume Joe Sheehan is, on a left-to-right spectrum, much closer to the "sabermetrics" end than he is the "traditionalist" end.

Here he is from 2004 - not exactly the swaddling days for sabermetrics - saying the very thing I argued against:



Am I misreading Sheehan here? Of course, I believe what the research says - but I don't believe they are all clutch. Relative to the population, yes - but I don't believe they all have the same mental makeup in these situations.

There's one not-too-antiquated quote. And here's the link.

Sheehan makes the point that there are zero players who have shown a statistically significant clutch ability. Bonds at one point was close to being statisticallly significant choker but then turned it around. For all of A-Rods well publicized failings his post-season career stats are remarkably close to his career stats.

Sheehan points out that the reason behind this is that if MLB players were not clutch they never would have gotten to the Major League level.

A legitimate point and one that I have never seen refuted by other than (bad) anecdotal evidence.

[spark240]

Originally Posted by NJYankeeFan

Sheehan points out that the reason behind this is that if MLB players were not clutch they never would have gotten to the Major League level.

Yes, players who make it to MLB are bound to be, as a group, more "clutch" than, say, average collegiate players. But that's not a useful statement. MLB players are also (as a group) faster runners, more powerful hitters, more accurate throwers, more sure fielders, better everything than any lesser group of players. Of course.

All of the distinctions that we make between this major leaguer and that one are going to be finer distinctions, more subtle differences than between either of them and Joe Schmo. They're still there, and in the specific environment of MLB, where everyone is (in the larger context) a superstar, those subtle differences in highly refined skills are magnified into significance.

[Tango Tiger]

I would not characterize the two articles by Sheehan and Neyer as representative of anything sabermetrics.

This is no different than a lawyer putting on the stand the two medical experts who he managed to find that is opposed to penicillin.

There is no uniformity of opinion regarding clutch, one way or the other, be it from sabermetricians, those who read about sabermetrics, those who write about sabermetrics, and those who have never heard of sabermetrics.

All I'm saying is not to use your paintbrush on all of us.

[JDD]

Originally Posted by ipitch

Isn't it kind of funny that these "clutch" players perform below their ability in less important situations? Can't they concentrate when the crowd is quiet?

Another thing that drives me nuts is comparing a particular player to "the norm". What is that? The norm of the league, or what we can reasonably expect from HIM?

If Wade Boggs only hit .304 on Tuesdays after the seventh inning in the month of April, is that bad? For him? Does that make him less than clutch? Or very clutch because the league will surely hit less than that?

I just don't see how you can suddenly become better whenever you sense a greater moment is going on around you. It is much easier to become nervous in that situation and turn into some player who is more likely to choke more often than what can be normally expected from that player.

I think it is less about rising to the occasion and more about not being overwhelmed by the moment. No research here, no fancy numbers, just a belief it is easier to dial up your level of concentration rather than suddenly improve your overall level of ability. This assumes the player is trying his very very best all the time.

[spark240]

Originally Posted by JDD

I just don't see how you can suddenly become better whenever you sense a greater moment is going on around you.

You don't? Have you ever worked on something important under pressure? How did you do?

Originally Posted by JDD

it is easier to dial up your level of concentration rather than suddenly improve your overall level of ability.

Well, maybe that's the key. The thing is, most people (ballplayers or otherwise, I suspect) can't "dial up" at will. It's more like something that happens, under certain circumstances, for some people.

[csh19792001]

Originally Posted by Tango Tiger

All data is meaningful. But, if your question is if the split is so meaningul that the split itself can be used without any regression, then the answer is 100% no.

Hi Tom,
Several questions....juxtaposed with you I'm a true neophyte here so please bear with me.

In your opinion, what threshold does a sample of behavior in cases like this become statistically significant (valid)? That is, I suppose, conclusions from which we can make summary conclusions that are valid and reliable.

Let's say a player came up as much as Pete Rose did in his career, with a career OPS+ of 100, and had an OPS+ of 120 in 3000 PA in "close and late" situations (let's assume those are the criteria we all agree on to define "clutch", for argument's sake). Obviously 300 PA isn't reliable, but is 3000? 5000?

Could we affix the qualitative description of "clutch" (assuming we all agree on the terminology)? How about the same hypo with "high leverage" situations and 3000 PA?

I also wonder about this with pitcher-batter matchups, especially given how much emphasis is placed on it by the media and by managers. Have you ever read "Three Nights in August"? Was Tom Glavine really just lucky against Brad Ausmus? Barry Bonds just got unlucky and his complete inability to hit Chuck McElroy was just random sample of behavior? If these guys got to face each other 250, 300 times how much would their performances progress to the mean?

If it turns out that Duke Snider's full career line is nearly identical to his 1953-65 performance, would you come to the conclusion that he really couldn't hit ML left handed pitching, or that he simply didn't face lefties enough?

[Pete Rose Rounding Third]

Tom:

Allow me to plead ignorance and ask: Where can I find more relevant summaries on the issue? Are there true sabermetricians who make an argument substantially different than those I posted by Neyer and Sheehan?

And I'm sorry I gave offense. I respect the hell out of the work that you and others do and I hope that, when I read other pieces from researchers, I can see for myself an acknowledgement that not all MLers have the same mental makeup.

I'm glad your work is out there. Without it, I'd have to hear a HOF 2Bman blather endlessly, without any counterpoint, to the effect that he can't evaluate guys he hasn't seen in person.

[brett]

Originally Posted by csh19792001

Hi Tom,
Several questions....juxtaposed with you I'm a true neophyte here so please bear with me.

In your opinion, what threshold does a sample of behavior in cases like this become statistically significant (valid)? That is, I suppose, conclusions from which we can make summary conclusions that are valid and reliable.

Let's say a player came up as much as Pete Rose did in his career, with a career OPS+ of 100, and had an OPS+ of 120 in 3000 PA in "close and late" situations (let's assume those are the criteria we all agree on to define "clutch", for argument's sake). Obviously 300 PA isn't reliable, but is 3000? 5000?

I am not expert here with the stats, but I will say two things. There are two or three different factors that are not mutually inclusive.

1) One is the chance of the plate appearance producing a run, ie men on base and outs.

2) Two is the chance of a run affecting the game outcome (which I believe is what "leverage" is, and depends only on inning and score, and perhaps the run environment).

3) Three is the degree to which hitters/pitcher feel pressure and this is critical if we want to call it a skill.

Now I would argue that some players with different skill sets say contact and batting average versus high walks may have a tendancy to produce more or less based on variations of factor 1. A contact hitter may be better with 2 outs and a runner on second or 1 out and a runner on third and since those situations are more likely to affect runs, those players may tend to be more valuable, but not due to any psychological effects.

Also, it seems clear to me that some hitters (Joe Morgan) are affected negatively by having a runner on second base but that this has nothing particularly to do with the pressure of the situation.

My second point will be that if 2000 plate appearances is not a large enough sample size, then neither is evaluating a player's performance over the course of a single season.

George Brett has a 113 relative OPS+ in high leverage situations. He also was "known" as a great clutch hitter. Now it does look like hitters as a group do a little better in high leverage situations (pitchers are more negatively affected?), but still a 113 relative OPS+ for 2000 plate appearances is clearly due to there being something different about Brett than an average hitter, but separating that into "skill set" variations, ability to perform under pressure, and situational/strategic factors (IBB's for example) is not easy to do.

Furthermore, Nolan Ryan allowing a 113 relative OPS+ with runners on base (almost half of his batters faced) is meaningful though he only allowed 89 late and close. The first is a measure of value. The second is a measure of pressure (and value) in part, and so he may have been good under felt pressure and just poor with men on base.

[Tango Tiger]

Originally Posted by csh19792001

Hi Tom,
Several questions....juxtaposed with you I'm a true neophyte here so please bear with me.

I think you'd appreciate The Book based on your questions that follow (free reading at Amazon.com).

In your opinion, what threshold does a sample of behavior in cases like this become statistically significant (valid)? That is, I suppose, conclusions from which we can make summary conclusions that are valid and reliable.

Roughly when you have 5000 or 6000 career PA, then your "true" clutch skill is halfway between your performance in that category to your overall performance.

In comparison, when you have 200 career PA, then your "true" overall skill batting is halfway between your performance in that category and that of the league.

I also wonder about this with pitcher-batter matchups, especially given how much emphasis is placed on it by the media and by managers. Have you ever read "Three Nights in August"? Was Tom Glavine really just lucky against Brad Ausmus? Barry Bonds just got unlucky and his complete inability to hit Chuck McElroy was just random sample of behavior? If these guys got to face each other 250, 300 times how much would their performances progress to the mean?

I don't know the exact answer, but it's going to be on the order of like clutch hitting.

If it turns out that Duke Snider's full career line is nearly identical to his 1953-65 performance, would you come to the conclusion that he really couldn't hit ML left handed pitching, or that he simply didn't face lefties enough?

If a LHH has 1000 PA against LHP, then his "true" "split" skill is halfway between his observed splits and the league splits.

This is all based on absence of other information. You can argue that Ichiro has reverse split skill, but you have to include the fact that this is ICHIRO and not some batter who happens to have those split stats.

[Tango Tiger]

Originally Posted by Pete Rose Rounding Third

Tom:

Allow me to plead ignorance and ask: Where can I find more relevant summaries on the issue? Are there true sabermetricians who make an argument substantially different than those I posted by Neyer and Sheehan?

And I'm sorry I gave offense. I respect the hell out of the work that you and others do and I hope that, when I read other pieces from researchers, I can see for myself an acknowledgement that not all MLers have the same mental makeup.

I'm glad your work is out there. Without it, I'd have to hear a HOF 2Bman blather endlessly, without any counterpoint, to the effect that he can't evaluate guys he hasn't seen in person.

I don't know where you can find relevant summaries. You can try the various wikis. And Cy Morong has a clutch webpage.

I appreciate the kind words, and I didn't take offense. Just making a point that we're individuals, and we don't all subscribe to the same gospel.

[barzilla]

A couple of salient points I'm not sure if anyone has contributed here, but when we look at clutch performance one of the issues is that you have two actors here. Could the pitcher also be a clutch performer or non-clutch performer? It seems to me that good pitchers are good because they are able to perform in certain situations just like those that look at a clutch hitter. So, where do we draw the line between clutch pitching and clutch hitting (or the reverse)?

Secondly, as I am sure Tango and Matt have done in their research, we need a baseline of average performance in any given situation. It is foolish to me to assume the average in all situations when any given situation might create an imbalance one way or the other. For instance, I would assume that hitters in general perform differently with men on base than with the bases empty. One of the definitions of clutch is with men on base. If we assume intuitively that a pitcher might have to pitch around the plate more often with men on base then we would expect OBP rates to change as well as BA and SLG rates. Of course, these are just considerations.

I've always been skeptical of measuring clutch simply because it is so hard to define which situations logically apply. As somone that is a sabermetrician in my spare time, I would say that I know it is exists, but simply we have come to a point in the field where we don't have universal agreement on how to measure it. I could be wrong and often am, but that is my sense of it.

[leewileyfan]

Clutch is increasingly diminished as a possibility with each restriction added to its defining qualifications:

-By some measures, any contest between KCR and CLE during Aug-Sept '09 would be tossed aside because there is little to nothing on the line between two non-contenders. Thus, heroics by David DeJesus, Jose Guillen, Asdrubial Cabrera or Shin-Soo Choo would go on the heap in any four game series between the two teams.

-However, in such contests and the personal struggles of a player in a rookie year; an option season; an adjustment to new surroundings; or playing one's heart out to get noticed for a possible trade away from a dead-end franchise.

Take, for example, the case of Jacoby Ellsbury during 2008. Manny departs and Jason Bay arrives. Coco Crisp is becoming a questionable fixture in CF; and Rocco Baldelli, if healthy, is back for another shot, perhaps also in CF. Every appearance is clutch in such surroundings; but I wouldn't begin to argue for that extent of generosity.

-Clutch articles and blogs, like Cyril Morongs's, allude to the principle of a clutch player performing above his non-pressure norm in "qualifying" clutch situations. IFthis were literally the standard, I'd have to toss all considerations of Pujols, Jeter, Guerrero, Bonds et. al. on the trash heap and declare that clutch doesn't exist. Why? Because a .340 hitter, .410 OB%, .550 SLG has too high a standard to measure up to.

Here are the points I'm exploring now [no deadline, no pressure] for yeat another perspective [for me] on clutch:

1. Start with MLB splits for season being considered, beginning with BA in situation, and including K% for each:

LG.....Overall AVG....... <0-1 Count........3B<2O............RISP

N.L.. .259; 20.8%... .228; 28.76%....319; 21.68%.. .265; 22.1%

A.L. .267; 19.63%.. .236; 26.48%....334; 21.08%.. .264; 20.8%

If I find similar splits for High Leverage AB, I'll add this as well. One entry I will use, NOT for BA but for K rate; and that is 3B<2O. In a similar study last year [never posted, just a personal exercise], this single observation revealed two famous players at opposite ends of the spectrum:

-Alfonso Soriano, whose high K rate rose in the 3B<2O situation;

-Albert Pujols, whose already low K rate fell considerably in the same situation.

There were others with telling numbers in that small category; and what it suggests to me is a distinction between thinking/aware hitters who adapt to situational demands, and those who mindlessly sing away in any situation [usually with predictably disappointing results].

For clutch to reveal itself, I suggest more inclusion & less exclusion.