This popped up on my facebook feed today, posted by OccupyDemocrats (of course,) and is an incredibly fabulous example of bad math.

Here's the WaPo link.

What WaPo says:

The finding comes from a government study considered a gold standard to measure public-health trends. Researchers found that just over 8 percent of children 2 to 5 were obese in 2011-2012, down from nearly 14 percent in 2003-2004. Although the drop was significant, federal health officials noted that obesity rates for the broader population remain unchanged, and for women older than 60, obesity rates rose about 21 percent during that period.

And here's the JAMA article link.

What JAMA says:

There was a significant decrease in obesity among 2- to 5-year-old children (from 13.9% to 8.4%; P = .03) 

So, did you see what Occupy did there? They took the rounded up 14% and the rounded down 8.4% and did this: (14 - 8)/(14) = 0.4285. Then claimed that was a decrease in obesity of 43%!!!

Ahem....no... It's: 13.9% - 8.4% =  5.5% drop in obesity among 2-5 year olds, not 43%.

However, here's another problem. Obama's big push has been to change the composition of school lunches, but the drop in obesity has been among pre-school aged kids. How exactly can she take credit for that?

Update: Apparently, this began with bad math from the New York Times back in February of 2014 in an article written by Sabrina Tavernise:

Federal health authorities on Tuesday reported a 43 percent drop in the obesity rate among 2- to 5-year-old children over the past decade, the first broad decline in an epidemic that often leads to lifelong struggles with weight and higher risks for cancer, heart disease and stroke.

[...] But the figures on Tuesday showed a sharp fall in obesity rates among all 2- to 5-year-olds, offering the first clear evidence that America’s youngest children have turned a corner in the obesity epidemic. About 8 percent of 2- to 5-year-olds were obese in 2012, down from 14 percent in 2004.

The article remains on their website, uncorrected, with the wording above (as of 7/6/15.)

Slate blasted the NYTimes piece  (corrected Salon to Slate. 15/08/23) back when it came out:

A far bigger issue is that studies like these, and the headlines that result, drive the discussion about public health and policy in this country. The media seizes on sexy results, amplifies them without due skepticism, and the public is misled. This can impact billions of dollars allocated to campaigns meant to capitalize on the supposed implications of scientific studies. It's hardly an academic footnote in this case. Commentators are already attempting to adduce the reasons for the decline in obesity in this age, pointing to the dietary changes in preschool menus, awareness campaigns, and exercise programs that specifically target tots.

Let \(\phi(x,\mu,\sigma)\) denote the normal distribution with mean \(\mu\) and standard deviation \(\sigma\) evaluated at \(x.\) Consider the model distribution $$p(x,\mu)=(1-t)\cdot \phi(x,\mu-3t,1)+t\cdot \phi(x,\mu+3\cdot(1-t),t^2),$$ for some known, fixed, but very small \(t>0.\) Since \(t\) is small, \(p\) looks very much like a normal distribution with variance \(1\) and mean \(\mu,\) except for a tall spike around \(\mu+3.\) See the picture below. The mean of \(p\) is \(\mu.\) Note in the last term, the variance is \(t^4.\) The total mass of the spike is \(t,\) thus small, and so the cumulative distribution for \(p\) will look very much like the standard cumulative distribution for \(\phi(x,\mu,1).\)

Suppose we have a sample \(x_0,\) assumed to be from a distribution \(p\) of the form above, but of unknown mean. To repeat, \(t\) is known. Consider the hypothesis: $$H_0: \mu=x_0-3\cdot(1-t)$$ Do we reject \(H_0\)?

If \(H_0\) is true, then \( p(x_0,\mu)\approx t \cdot \phi(x_0,x_0,t^2) = 1/(t \sqrt{2\pi}),\) and this is very large, about \(1/t\) times larger than \(p(\mu,\mu).\) Indeed, \(x_0-3\cdot(1-t)\) is very close to the maximum likelihood estimate for \(\mu.\) These seem to be reasons not to reject \(H_0.\)

On the other hand, the overall mass of the spike is small, and the entire spike is well out on the tail of \(p.\) Thus it is in some sense unlikely to get an \(x_0\) out there, assuming \(H_0\) is true. Since the cumulative distribution looks very much like the normal cumulative distribution for \(\phi(x,\mu,1),\) it makes sense to apply the usual test and reject \(H_0.\)

I find that not rejecting in this case is the right thing to do, but I am not sure what others might think. I also wonder how often such a question is relevant.

 

My random thoughts on random events posted as a blog comment:

Suppose we have disjoint events \( (E_1,E_2,\dots,E_N),\) and corresponding probabilities for these events \( (p_1,p_2,\dots,p_N),\) where \( p_n=\mbox{Prob}(E_n)\) and \(\sum_{n=1}^N p_n=1.\) If a particular event \(E_k\) occurs, what would make us think this was in some sense "unusual" or perhaps "suspicious"? It's not enough that \(p_k\) be small, since for large \(N\), even a uniform distribution on the \(E_n\) will have \(p_k=1/N\) small. Nor is it enough that \(p_k\) be much less than \( \max_n p_n,\) since it is possible that all \(p_n\) are equal except for one event having many times larger yet still tiny probability. It's not enough if \(p_k\) is less than nearly all the other \(p_n\), because all the \(p_n\) could be very nearly equal.

What does seem to work in the cases I can think of is to choose some factor \(R \gt 1\), calculate \(\sum\{p_n: p_n \gt R p_k\}\), and see if this is close to \(1.\) To work this into a hypothesis test, we could reject the null hypothesis \(H_0\) if $$\sum\{p_n: p_n\gt R p_k\} \gt (1-1/R),$$ though the expression on the right-hand side is rather arbitrary. With this setup, what value should \(R\) be? Let \(x_0\) be a sample we have collected, and consider the standard normal and the \(p=.05\) rule, where \(\mbox{Prob}(|x_0|\gt 1.96)=0.05.\) Then \(R=3.71,\) since \(3.71\cdot \phi(1.96) = \phi(1.1),\) and \(\mbox{Prob}(|x_0|\gt 1.1)=1/3.71.\) If we wanted \(R=20,\) we would need to use a cutoff \( |x_0|\gt 3.135749,\) which corresponds to a very small standard \(p\)-value of \(0.001714.\)

Clearly, given any \(p\) cutoff, a.k.a \(\alpha\), we can find a corresponding factor \(R,\) and vice-versa. Since the \(p=.05\) rule is arbitrary, I don't see what difference it makes for the most common cases. Thus, \(p\)-value analysis seems generally ok to me in practice. My concern here is with its justification.

Here is a figure from some notes I am reading:

My understanding of it is that we have a stiff cube-shaped frame, say made out of wooden dowels or thick wire, fixed in space. In the center of the frame we have a ball, attached to the corners of the frame with 8 rubber bands.

The claim is that if we rotate the ball 360 degrees around the vertical axis, and hold it there, we would be unable to untangle the bands without moving the ball. However, if we rotate the ball 720 degrees around the vertical axis and hold it there, then we could get the bands back how they were when we started.

Wow. Can you see in your head how this works? I'm tempted to build one. It seems weird to me. At what degree of rotation does this untangling become possible? Supposedly it's more than 360 and less than or equal to 720 degrees. How about 719 degrees? How does this magically jump from "not possible" to "possible" as we continuously rotate the ball?

Sometimes, after solving a problem, it seems to me there should be a symmetry argument that I missed.

You have a cup of coffee and a cup of cream. Put a teaspoon of the cream in the coffee and stir. Take a teaspoon of this mixture and put it in the cream. Is there now more cream in the coffee or coffee in the cream?

Neither. It turns out that they are equal. Should this have been obvious? Why? Is there a symmetry argument?

Ann says: Man! that took me a while to figure out. I forgot that the volume of the coffee before transferring back to the milk is (T + 1)/T. So in the end, each cup has 1/(T + 1) of the other?

I think that's right. If the spoon holds T cups, then in the end the coffee cup contains 1/(T+1) cups of coffee and T/(T+1) cups of cream. Vice versa for the cup of cream. You can stick in T=0 and T=1 and see that the formulas make sense. After doing the math though, it seems that there should be some way to see intuitively why the equality is there.

Benoît Mandelbrot passed away recently. He was a mathematician known for his work with fractals. Indeed, he coined the term. I saw him speak once, and I got the impression that he was a bit of a kook. Whatever my impression, there is no question that his work was of great influence.

Here are some beautiful renderings of 3D fractal sets. Be sure to scroll down the page for the best ones. Quite spectacular.

RIP.

Interesting... many of the bits I've seen from Everyday Math are similar to the way I do arithmetic in my head. For 754-472, I would add 200+80+2.

Ann adds: I spontaneously did what I call "adding-up" subtraction a few years ago. I put absolutely no thought into it, but suddenly found myself looking at the bottom number, figuring out how many to add to get to the upper number, then, if I had to borrow, borrowing by adding to the bottom, instead of subtracting from the top. Instead of crossing off and rewriting above the problem--leaving everything a mess, all this method requires is a little tick mark on the summation line to signal when you have to add to the bottom number. I've been afraid to teach that to the kids, because (unlike EM), I think they should use just one algorithm, and get that one down, and not jump between a lot of different methods.

So, for your example, I'd ad 2 to get to 4, then, since I can't add to get from 7 to 5, I put a tick next to the bottom 4 and add from 7 to 15. Then add the tick mark to the 4 to turn it into a 5, and add 2 to get from 5 to 7. It's a really clean and easy way to do it.

Where EM really falls down is the inordinate time spent doing the "lattice method" for multiplication. I think they spend better than half of their multiplication time on it. And, when you start working with really large numbers, the method gets really tough to use.

Since John Derbyshire has posted code for the probability problem, I thought I'd provide my own, written in C. Typical output is:

Two boys: 2499636 0.249964
One boy, one girl: 4999669 0.499967
Two girls: 2500694 0.250069

Tuesday boys: 714025 0.071402
Their brothers: 357115 0.500144
Their sisters: 356910 0.499856
Enjoy!

#include <stdlib.h>
#include <stdio.h>
#include <math.h>

#define GIRL 1
#define BOY 2
#define TUESDAY 3

/* return a random integer from 1 to max */
int randint(int max)
  {
  return(1+floor(max*(rand()/(1+(double)RAND_MAX))));
  }

int main(void)
  {
  int N=10000000; /* number of trials */

  int n,g[3],d[3],a,b;
  int twogirls=0,twoboys=0,boygirl=0;
  int tuesdayboys=0,sibgirls=0,sibboys=0;

  for (n=1;n<=N;n++)
    {
    g[1]=randint(2);  /* random gender of child 1 (GIRL==1, BOY==2) */
    d[1]=randint(7);  /* random birth day of child 1 (SUNDAY==1, SATURDAY==7) */

    g[2]=randint(2);  /* random gender of child 2 */
    d[2]=randint(7);  /* random birth day of child 2 */
  
    a=randint(2);  /* child parent tells you about (1 or 2), chosen at random */
    b=3-a;         /* child parent tells you nothing about (1 or 2) */

    if ((g[a]==BOY)&&(g[b]==BOY)) twoboys++;   /* increase count of 2-boy families by one */
    if ((g[a]==BOY)&&(g[b]==GIRL)) boygirl++;  /* increase count of 1-boy 1-girl families by one */
    if ((g[a]==GIRL)&&(g[b]==BOY)) boygirl++;  /* increase count of 1-boy 1-girl families by one */
    if ((g[a]==GIRL)&&(g[b]==GIRL)) twogirls++; /* increase of 2-girl families by one */
    
    if ((g[a]==BOY)&&(d[a]==TUESDAY)) /* if the child the parent chooses to tell you about is a Tuesday-boy */
      {
      tuesdayboys++; /* increase count of Tuesday-boys you are told about by one */
      if (g[b]==BOY) sibboys++;  /* increase count of their brothers by one */
      if (g[b]==GIRL) sibgirls++; /* increase count of their sisters by one */
      }
    }

  /* print out counts and ratios */
  printf("         Two boys: %d %f\n",twoboys,twoboys/(double)N);
  printf("One boy, one girl: %d %f\n",boygirl,boygirl/(double)N);
  printf("        Two girls: %d %f\n",twogirls,twogirls/(double)N);
  printf("\n");
  printf("     Tuesday boys: %d %f\n",tuesdayboys,tuesdayboys/(double)N);
  printf("   Their brothers: %d %f\n",sibboys,sibboys/(double)tuesdayboys);
  printf("    Their sisters: %d %f\n",sibgirls,sibgirls/(double)tuesdayboys);

  return(0);
  }

#undef TUESDAY
#undef BOY
#undef GIRL

There is an interesting probability problem floating around the net. It is easily stated:

"I have two children. One is a boy born on a Tuesday. What is the probability I have two boys?"

The "convential wisdom," from the math experts, is that the answer is 13/27th, not the 1/2 that one might expect. I read about this problem here and here and here, the last link being to John Derbyshire's blog. He has comments in the corner too, here, here, and here.

I came up with the same solution, but later had my doubts. I now side with the folks who say the answer is 1/2. The problem is one of language, which is ambiguous.

When someone says "I have two children. One is a boy born on a Tuesday" does he mean that he has at least one Tuesday-born boy, or does he mean that one specific kid, call him William, was born on a Tuesday? If the former, the answer is 13/27. If the latter, the answer is 1/2. That's the ambiguity.

Consider another, simpler, scenario: "I have two children. One is a boy." Has he told you that at least one of his kids is a boy? If so, then the probability of two boys is 1/3. Has he told you the gender of one specific kid chosen at random? If so, the two-boy probability is 1/2. What do you think is the more common meaning of the statement? I feel pretty sure that the latter interpretation, yielding 1/2, is far more often correct. This is not the interpretation of the "experts" linked above, who would give the answer as 1/3.

A comment posted here agrees with me and is a good explanation. Another great comment here.

Fairly regularly, I read on the web people stating emphatically that:

"You can't prove a negative!"

Why not? If A is provable, and we let B=~A, then clearly the negative ~B is provable, because ~B=~(~A)=A.

It is also possible to prove “universal negatives” deductively, depending on the universe. For example, it is possible to prove that there does not exist a solution to exp(x)=0 in the universe of the real (or complex) number system.

As a non-mathematical example, the negative statement “there does not exist a two ton elephant in my dishwasher” seems pretty provable to me too, where “pretty provable” might mean there is a very high likelihood. I could simply weigh the dishwasher along with its contents. If it weighs less than two tons, then I think that pretty much proves it.

Surely some statements are more easily proved than others. "I did not sneeze yesterday" is tough to prove. But so is "I sneezed yesterday." Provability has nothing to do with whether or not the statement is "positive" or "negative." It is not even clear to me what it means for a statement to be "positve" or "negative".

I think the way hypothesis testing is presented to students and justified makes no sense. Suppose we know that x has been drawn from a normal distribution with unit variance, and we want to test H_0: mu=0. Suppose x=2. Then they say “given H_0, the chance of seeing |x|>1.96 is less than 5%, so since x=2>1.96 we reject H_0.” Why does this make sense? Given H_0, the chance of seeing |x|<.01 is less than 5% too. Would you reject the null if our sample were x=0? No! So then they talk about x being “extreme,” i.e. far from the mean. What exactly does distance from the mean have to do with it? Suppose we knew that x was drawn from a uniform distribution on some interval [mu-1/2,mu+1/2], and again we wanted to test H0: mu=0. If x=.4999 would you reject? No, that makes no sense, because given H0, x=.4999 is no less likely than x=0. You could reject if x=.5001, but not if x is in the interval (-1/2,1/2). You could easily find an example of a bimodal distribution where the pdf at the mean is zero. Then you should reject if the sample is near the mean! Distance from the mean is not in general relevant.

I am being pedantic, but hypothesis testing works e.g. for the standard normal distribution f because if x>1.96, then f(x) is much less than values of f near x=0, not because of areas at the tails or distance from the mean.

President Obama yesterday:

Now, so let me talk about the third thing, which is my proposal would bring down the cost of health care for families, for businesses, and for the federal government. So Americans buying comparable coverage to what they have today -- I already said this -- would see premiums fall by 14 to 20 percent -- that’s not my numbers, that’s what the nonpartisan Congressional Budget Office says -- for Americans who get their insurance through the workplace. How many people are getting insurance through their jobs right now? Raise your hands. All right. Well, a lot of those folks, your employer it’s estimated would see premiums fall by as much as 3,000 percent [sic], which means they could give you a raise. (Applause.)

Wow. Just... WOW. If premiums fell to $0 that would be a 100% decrease. This is 30 times more than that. That our president suffers from this level of innumeracy is shocking and deeply disturbing.

Related to a climate statistics post here.

function [tau,mu]=roman3(x,h,w);
% Finds tau and mu to minimize the sum of squares
% \sum_n \sum_m h(m,n)*(x(m,n)-tau(n)-mu(m))^2,
% subject to the constraint \sum_m w(m)*mu(m)=0. Here x and h are MxN,
% and w is Mx1. Normally, one would have non-negative h, positive w, and
% \sum_m w(m)=1. Upon return, tau will be 1xN, mu will be Mx1.
% The method uses a Lagrange multiplier (lambda) and Gaussian elimination.
% The matrix h must not contain an all-zero row or column.

[M,N]=size(x); % x and h have M rows, N columns
A=zeros(M+N+1,M+N+1); % A will contain the linear system matrix
A(1:N,1:N+M)=[diag(sum(h,1)) h']; % dL/dtau, N equations
A(N+1:N+M,:)=[h diag(sum(h,2)) -w/2]; % dL/dmu, M equations
A(end,N+1:N+M)=w'; % dL/dlambda, one equation (the constraint)
b=[sum(h.*x,1)'; sum(h.*x,2); 0]; % b is the "right hand side"
y=A\b; % solve Ay=b for y, using Gaussian elim.
tau=y(1:N)';
mu=y(N+1:N+M); % note the multiplier lambda is always zero
return

Ann says: Man, you beat me to it. That's just what I was thinking!

William M Briggs mentions Laplace's Rule of Succession in a recent blog post. Briggs' is a blog about statistics and related matters that I highly recommend. Laplace used the rule, which relies on Bayes' Theorem, to calculate the probability that the sun will rise tomorrow. It is an elegant and fascinating bit of analysis. According to Wikipedia, Laplace's method give odds 1826250:1 in favour of the sun rising tomorrow.

But I beg to differ! Using Bayesian analysis, I calculate that the probability that the sun will rise tomorrow is 1/2!

Here is my argument. Let N+1 be the number of days from the start of history through tomorrow. Suppose at first that we know nothing about the sun, neither the related physics nor the past history of its rising. This was Laplace's assumption as well. Suppose we only know that the sun rises on some subset of the N+1 days in question. With only this knowledge, we assume a uniform prior probability distribution on this subset. Thus we assume that all 2^(N+1) possible subsets of the N+1 days are equally likely to be the sun-rising subset, each having probability 2^(-N-1). Now suppose we are given additional knowledge, specifically that the sun rose on the first N days. There are then only two possibilities for the sun-rising subset: the set of all N+1 days, and the set that contains only the first N days. By our prior assumption, and a trivial application of Bayes Rule, we see that each of these possibilities now has the posterior probability of 1/2. Thus the probability of the sun rising tomorrow is 1/2.

Put that in your pipe and smoke it! Related post here.

I was reading the article on Bayes' Theorem on Wikipedia, which currently contains this bit on Bayesian inference:

Suppose we wish to know about the proportion r of voters in a large population who will vote "yes" in a referendum. Let n be the number of voters in a random sample (chosen with replacement, so that we have statistical independence) and let m be the number of voters in that random sample who will vote "yes". Suppose that we observe n = 10 voters and m = 7 say they will vote yes. From Bayes' theorem we can calculate the probability distribution function for r using

From this we see that from the prior probability density function f(r) and the likelihood function L(r) = f(m = 7r, n = 10), we can compute the posterior probability density function f(rn = 10, m = 7).

The prior probability density function f(r) summarizes what we know about the distribution of r in the absence of any observation. We provisionally assume in this case that the prior distribution of r is uniform over the interval [0, 1]. That is, f(r) = 1. If some additional background information is found, we should modify the prior accordingly. However before we have any observations, all outcomes are equally likely. [Emphasis added]

This last bit sticks in my craw, and I'll say why below. The article goes on to do some standard calculations, ending up with the conclusion:

One may be interested in the probability that more than half the voters will vote "yes". The prior probability that more than half the voters will vote "yes" is 1/2, by the symmetry of the uniform distribution. In comparison, the posterior probability that more than half the voters will vote "yes", i.e., the conditional probability given the outcome of the opinion poll – that seven of the 10 voters questioned will vote "yes" – is [.887], which is about an "89% chance".

Think for a moment about that conclusion. Is it reasonable? The experiment had 7 out of the 10 people asked saying they would vote 'yes.' Even if the total voting population is several million people, our new estimate of the probability that the referendum will pass is supposedly 89%. It doesn't seem to me that we have nearly enough evidence to reach that sort of conclusion. Mind you, I think the calculations based on Bayes' theorem are correct, and from my experience, similar calculations are common by practitioners of Bayesian inference.

But I disagree strongly with the conclusion. How is that possible? Math doesn't lie. The problem is with the assumption, which I italicized above

However before we have any observations, all outcomes are equally likely.

Even if you buy the idea that all outcomes are equally likely, perhaps justifying this assumption using a maximum entropy argument, there is the fundamental question of what an "outcome" is. I can easily choose a different assumption, no less justifiable, which results in a drastically different conclusion. It seems to me that if we really know nothing about the subset of voters who will be voting 'yes', then we might assume a prior on this 'yes'-subset as uniform on the collection of all subsets of the population. If there are N voters, then there are 2^N subsets, each of which might occur with an equal probability of 2^(-N). This uniform prior on the 'yes'-subset induces a prior on the rate r. For large N, the binomial theorem can be used to show that the prior f(r) is essentially a normal distribution with mean 0.5 and tiny standard deviation (4*N)^(-1/2). In words, practically all subsets of a large population N contain roughly N/2 elements.

Given this alternative prior, and the outcome of the survey, what is the new estimate of probability that the referendum passes? It's still essentially 50%. You could go through the exact calculation, but that's the answer you will get. With this prior, the evidence obtained from the survey of 10 people is nearly meaningless.

So who is right and who is wrong in his or her assumptions? No one can say. The fact is that this sort of blind inference is based on arbitrary assumptions, and without more information, there is literally no way to know.

John Derbyshire serves up a math problem in his August Diary:

Math Corner. A reader sent this in:

You can flip coins until the number of heads and tails are equal. The payoff is the number of flips. So, half of all games end after two flips and pay two dollars …

I'm not sure how you make a betting game out of this, but I'm certainly willing to tackle the question: What proportion of games, on average, end after exactly T coin tosses? Pretty obviously T has to be an even number. The chance that a game will end after two tosses is of course, as my reader says, 0.5. A couple of minutes' doodling should convince you that for four tosses the answer is 0.125. That is, one-eighth of games on average will end on the fourth toss. And then …?

Now once upon a time I might have been tempted to try to work this out for myself. Maybe I would have figured it out. Maybe not. However, with Google around, I often find myself just looking up the answer to such problems.

You just have to know the right question to ask! Googling "random walk return time" gives as the first item a pdf article with the answer near the bottom of page 2:

Prob(game ends on 2k flips)=C(2k,k)(1/4)^(k)/(2k-1)

where C(n,k) is the well-known function "n choose k."

The mathematical answer to the guy who wrote Mr. Derbyshire is that the expected payoff is infinite. This is stated at the end of section 1.1. In theory you should be willing to bet all your money on this game (*). It seems counter-intuitive; that's probably the point of the question.

I give myself credit for Googling skills, but subtract for general laziness. I wonder to what extent school kids today get their homework done via Google.

(*) On second thought, this isn't necessarily so. You'd have to take into account your risk aversion.