Foundations of Statistics for Data Scientists : With R and Python (Hardcover)

　 Table of Contents: Foundations of Statistical Science for Data Scientists Alan Agresti and Maria Kateri 1. Introduction to Statistical Science 1.1 Statistical science: Description and inference Design, descriptive statistics, and inferential statistics Populations and samples Parameters: Numerical summaries of the population Defining populations: actual and conceptual 1.2 Types of data and variables Data files Example: The General Social Survey (GSS) Variables Quantitative variables and categorical variables Discrete variables and continuous variables Associations: response variables and explanatory variables 1.3 Data collection and randomization Randomization Collecting data with a sample survey Collecting data with an experiment Collecting data with an observational study Establishing cause and effect: observational versus experimental studies 1.4 Descriptive statistics: Summarizing data Example: Carbon dioxide emissions in European nations Frequency distribution and histogram graphic Describing the center of the data: mean and median Describing data variability: standard deviation and variance Describing position: percentiles, quantiles, and box plots 1.5 Descriptive statistics: Summarizing multivariate data Bivariate quantitative data: The scatterplot, correlation, and regression Bivariate categorical data: Contingency tables Descriptive statistics for samples and for populations 1.6 Chapter summary Exercises 2. Probability Distributions 2.1 Introduction to probability Probabilities and long-run relative frequencies Sample spaces and events Probability axioms and implied probability rules Example: Diagnostics for disease screening Bayes' theorem Multiplicative law of probability, and independent events 2.2 Random variables and probability distributions Probability distributions for discrete random variables Example: Geometric probability distribution Probability distributions for continuous random variables Example: Uniform distribution Probability functions (pdf, pmf) and cumulative distribution function (cdf) Example: Exponential random variable Families of probability distributions indexed by parameters 2.3 Expectations of random variables Expected value and variability of a discrete random variable Expected values for continuous random variables Example: Mean and variability for uniform random variable Higher moments: Skewness Expectations of linear functions of random variables Standardizing a random variable 2.4 Discrete probability distributions Binomial distribution Example: Hispanic composition of jury list Mean, variability, and skewness of binomial distribution Example: Predicting results of a sample survey The sample proportion as a scaled binomial random variable Poisson distribution Poisson variability and overdispersion 2.5 Continuous probability distributions The normal distribution The standard normal distribution Examples: Finding normal probabilities and percentiles The gamma distribution The exponential distribution and Poisson processes Quantiles of a probability distribution Using the uniform to randomly generate a continuous random variable 2.6 Joint and conditional distributions and independence Joint and marginal probability distributions Example: Joint and marginal distributions of happiness and family income Conditional probability distributions Trials with multiple categories: the multinomial distribution Expectations of sums of random variables Independence of random variables Markov chain dependence and conditional independence 2.7 Correlation between random variables Covariance and correlation Example: Correlation between income and happiness Independence implies zero correlation, but not converse Bivariate normal distribution * 2.8 Chapter summary Exercises 3. Sampling Distributions 3.1 Sampling distributions: Probability distributions for statistics Example: Predicting an election result from an exit poll Sampling distribution: Variability of a statistic's value among samples Constructing a sampling distribution Example: Simulating to estimate mean restaurant sales 3.2 Sampling distributions of sample means Mean and variance of sample mean of random variables Standard error of a statistic Example: Standard error of sample mean sales Example: Standard error of sample proportion in exit poll Law of large numbers: Sample mean converges to population mean Normal, binomial, and Poisson sums of random variables have the same distribution 3.3 Central limit theorem: Normal sampling distribution for large samples Sampling distribution of sample mean is approximately normal Simulations illustrate normal sampling distribution in CLT Summary: Population, sample data, and sampling distributions 3.4 Large-sample normal sampling distributions for many statistics * Delta method Delta method applied to root Poisson stabilizes the variance Simulating sampling distributions of other statistics The key role of sampling distributions in statistical inference 3.5 Chapter summary Exercises 4. Statistical Inference: Estimation 4.1 Point estimates and confidence intervals Properties of estimators: Unbiasedness, consistency, efficiency Evaluating properties of estimators Interval estimation: Confidence intervals for parameters 4.2 The likelihood function and maximum likelhood estimation The likelihood function Maximum likelihood method of estimation Properties of maximum likelihood estimators Example: Variance of ML estimator of binomial parameter Example: Variance of ML estimator of Poisson mean Sufficiency and invariance for ML estimators 4.3 Constructing confidence intervals Using a pivotal quantity to induce a confidence interval A large-sample confidence interval for the mean Confidence intervals for proportions Example: Atheists and agnostics in Europe Using simulation to illustrate long-run performance of CIs Determining the sample size before collecting the data Example: Sample size for evaluating an advertising strategy 4.4 Confidence intervals for means of normal populations The $t$ distribution Confidence interval for a mean using the $t$ distribution Example: Estimating mean weight change for anorexic girls Robustness for violations of normal population assumption Construction of $t$ distribution using chi-squared and standard normal Why does the pivotal quantity have the $t$ distribution? Cauchy distribution: t distribution with df=1 has unusual behavior 4.5 Comparing two population means or proportions A model for comparing means: Normality with common variability A standard error and confidence interval for comparing means Example: Comparing a therapy to a control group Confidence interval comparing two proportions Example: Does prayer help coronary surgery patients? 4.6 The bootstrap Computational resampling and bootstrap confidence intervals Example: Confidence intervals for library data 4.7 The Bayesian approach to statistical inference Bayesian prior and posterior distributions Bayesian binomial inference: Beta prior distributions Example: Belief in hell Interpretation: Bayesian versus classical intervals Bayesian posterior interval comparing proportions Highest posterior density (HPD) posterior intervals 4.8 Bayeian inference for means Bayesian inference for a normal mean Example: Bayesian analysis for anorexia therapy Bayesian inference for normal means with improper priors Predicting a future observation: Bayesian predictive distribution The Bayesian perspective, and empirical Bayes and hierarchical Bayes extensions 4.9 Why maximum likelihood and Bayes estimators perform well * ML estimators have large-sample normal distributions Asymptotic efficiency of ML estimators same as best unbiased estimators Bayesian estimators also have good large-sample performance The likelihood principle 4.10 Chapter summary Exercises 5. Statistical Inference: Significance Testing 5.1 The elements of a significance test Example: Testing for bias in selecting managers Assumptions, hypotheses, test statistic, P-value and conclusion 5.2 Significance tests for proportions and means The elements of a significance test for a proportion Example: Climate change a major threat? One-sided significance tests The elements of a significance test for a mean Example: Significance test about political ideology 5.3 Significance tests comparing means Significance tests for the difference between two means Example: Comparing a therapy to a control group Effect size for comparison of two means Bayesian inference for comparing means Example: Bayesian comparison of therapy and control groups 5.4 Significance tests comparing proportions Significance test for the difference between two proportions Example: Comparing prayer and non-prayer surgery patients Bayesian inference for comparing two proportions Chi-squared tests for multiple proportions in contingency table Example: Happiness and marital status Standardized residuals: Describing the nature of an association 5.5 Significance test decisions and errors The alpha-level: Making a decision based on the P-value Never ``accept H_0'' in a significance test Type I and Type II errors As P(Type I error) decreases, P(Type II error) increases Example: Testing whether astrology has some truth The power of a test Making decisions versus reporting the P-value 5.6 Duality between significance tests and confidence intervals Connection between two-sided tests and confidence intervals Effect of sample size: Statistical versus practical significance Significance tests are less useful than confidence intervals Significance tests and P-values can be misleading 5.7 Likelihood-ratio tests and confidence intervals * The likelihood-ratio and a chi-squared test statistic Likelihood-ratio test and confidence interval for a proportion Likelihood-ratio, Wald, score test triad 5.8 Nonparametric tests A permutation test to compare two groups Example: Petting versus praise of dogs Wilcoxon test: Comparing mean ranks for two groups Comparing survival time distributions with censored data 5.9 Chapter summary Exercises 6. Linear Models and Least Squares 6.1 The linear regression model and its least squares fit The linear model describes a conditional expectation Describing variation around the conditional expectation Least squares model fitting Example: Linear model for Scottish hill races The correlation Regression toward the mean in linear regression models Linear models and reality 6.2 Multiple regression: Linear models with multiple explanatory variables Interpreting effects in multiple regression models Example: Multiple regression for Scottish hill races Association and causation Confounding, spuriousness, and conditional independence Example: Modeling the crime rate in Florida Equations for least squares estimates in multiple regression Interaction between explanatory variables in their effects Cook's distance: Checking for unusual observations 6.3 Summarizing variability in linear regression models The error variance and chi-squared for linear models Decomposing variability into model explained and unexplained parts R-squared and the multiple correlation Example: R-squared for modeling Scottish hill races 6.4 Statistical inference for normal linear models The F distribution: Testing that all effects equal 0 Example: Linear model for mental impairment t tests and confidence intervals for individual effects Multicollinearity: Nearly redundant explanatory variables Confidence interval for E(Y) and prediction interval for Y The F test that all effects equal 0 is a likelihood-ratio test * 6.5 Categorical explanatory variables in linear models Indicator variables for categories Example: Comparing mean incomes of racial-ethnic groups Analysis of variance (ANOVA): An F test comparing several means Multiple comparisons of means: Bonferroni and Tukey methods Models with both categorical and quantitative explanatory variables Comparing two nested normal linear models Interaction with categorical and quantitative explanatory variables 6.6 Bayesian inference for normal linear models Prior and posterior distributions for normal linear models Example: Bayesian linear model for mental impairment Bayesian approach to the normal one-way layout 6.7 Matrix formulation of linear models The model matrix Least squares estimates and standard errors The hat matrix and the leverage Alternatives to least squares: Robust regression and regularization Restricted optimality of least squares: Gauss--Markov theorem Matrix formulation of Bayesian normal linear model 6.8 Chapter summary Exercises 7. Generalized Linear Models 7.1 Introduction to generalized linear models The three components of a generalized linear model GLMs for normal, binomial, and Poisson responses Example: GLMs for house selling prices The deviance Likelihood-ratio model comparison uses deviance difference Model selection: AIC and the bias/variance tradeoff Advantages of GLMs versus transforming the data Example: Normal and gamma GLMs for Covid-19 data 7.2 Logistic regression for binary data Logistic regression: Model expressions Interpreting beta_j: effects on probabilities and odds Example: Dose-response study for flour beetles Grouped and ungrouped binary data: Effects on estimates and deviance Example: Modeling Italian employment with logit and identity links Complete separation and infinite logistic parameter estimates 7.3 Bayesian inference for generalized linear models Normal prior distributions for GLM parameters Example: Bayesian logistic regression for endometrial cancer patients7.4 Poisson loglinear models for count data Poisson loglinear models Example: Modeling horseshoe crab satellite counts Modeling rates: Including an offset in the model Example: Lung cancer survival 7.5 Negative binomial models for overdispersed count data * Increased variance due to heterogeneity Negative binomial: Gamma mixture of Poisson distributions Example: Negative binomial modeling of horseshoe crab data 7.6 Iterative GLM model fitting * The Newton--Raphson method Newton--Raphson fitting of logistic regression model Covariance matrix of parameter estimates and Fisher scoring Likelihood equations and covariance matrix for Poisson GLMs 7.7 Regularization with large numbers of parameters Penalized likelihood methods Penalized likelihood methods: The lasso Example: Predicting opinions with student survey data Why shrink ML estimates toward 0? Dimension reduction: Principal component analysis Bayesian inference with a large number of parameters Huge n: Handling big data 7.8 Chapter summary Exercises % 8. Classification and Clustering 8.1 Classification: Linear Discriminant Analysis and Graphical Trees Classification with Fisher's linear discriminant function Example: Predicting whether horseshoe crabs have satellites Summarizing predictive power: Classification tables and ROC curves Classification trees: Graphical prediction Logistic regression versus linear discriminant analysis and classification trees Other methods for classification: k-nearest neighbors and neural networks prediction 8.2 Cluster Analysis Measuring dissimilarity between observations on binary responses Hierarchical clustering algorithm and its dendrogram Example: Clustering states on presidential election outcomes 8.3 Chapter summary Exercises 9. Statistical Science: A Historical Overview 9.1 The evolution of statistical science Evolution of probability Evolution of descriptivev and inferential statistics 9.2 Pillars of statistical wisdom and practice Stigler's seven pillars of statistical wisdom Seven pillars of wisdom for practicing data science Appendix A: Using R in Statistical Science Appendix B: Using Python in Statistical Science Appendix C: Brief Solutions to Odd-Numbered Exercises Bibliography Example Subject Index