Sublime

However, instead of the traditional hypothesis-based approach to statistical analysis, in which the analyst or decision maker comes up with a hypothesis and then tests it for fit with the data, big data analysis is more likely to involve machine learning.

nonparametric methods are better suited to small samples. There’s no point using nonparametric statistics if you’ve got a lot of data (for example n > 100).

Parametric statistics is a branch of statistics that assumes sample data come from populations that are adequately modeled by probability distributions with a set of parameters.

If we add and subtract the margin of error to the sample mean of 80, we have a 95% confidence interval that ranges from 67.6 to 92.4, which, as expected, contains the population mean of 78 (see Chapter 3 for more detail on generating confidence intervals).

such as the Kolmogorov-Smirnov test, the meaning of which, I have to admit, I had forgotten. (It’s a way to determine whether a model correctly fits data.)

The null hypothesis is rejected if your data set is unlikely to have been produced by chance. The significance of the results is described by the confidence level that was defined by the test (as described by the acceptable error “alpha-level”).

The fact is that, despite its mathematical base, statistics is as much an art as it is a science. A great many manipulations and even distortions are possible within the bounds of propriety.