Core Science – Data Analysis
M.Tech. Core Science (Data Analysis)
A concise, exam‑oriented study guide (IIT‑JEE / competitive) with derivations, key formulae, tricky points, and a hard practice question.
1. Overview
Data analysis in a scientific context is the systematic process of collecting, cleaning, summarising, model‑building, and drawing inference from data.
The core concepts are:
| Domain | Typical Topics | Key Idea |
|---|---|---|
| Descriptive Statistics | Mean, Median, Mode, Variance, Skewness, Kurtosis, Box‑plots | Summarise data |
| Probability Distributions | Binomial, Poisson, Normal, Exponential, Gamma, t, χ², F | Model random phenomena |
| Sampling & Estimation | Sampling distribution, Central Limit Theorem (CLT), bias, variance, confidence intervals | Infer population from sample |
| Hypothesis Testing | Null/Alternative, Type‑I/II errors, p‑value, z, t, χ², ANOVA | Decide between competing hypotheses |
| Regression & Correlation | Linear regression, multiple regression, R², hypothesis tests on coefficients | Predict / explain relationships |
| Non‑parametric Methods | Mann–Whitney, Kruskal–Wallis, Spearman | Handle ordinal or non‑normal data |
| Bayesian Inference | Prior, Likelihood, Posterior, Bayes’ theorem | Update beliefs with evidence |
| Multivariate Analysis | PCA, Factor analysis, Cluster analysis | Reduce dimensionality / group data |
Exam tip:
*Know the assumptions of each test (normality, independence, equal variances).
*Know the formulae for test statistics and critical values.
Be ready to derive a test statistic from first principles.
2. Descriptive Statistics
| Symbol | Definition | Formula |
|---|---|---|
| Sample mean (\bar{x}) | Arithmetic average | (\displaystyle \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i) |
| Sample variance (s^2) | Average squared deviation | (\displaystyle s^2=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2) |
| Sample standard deviation (s) | (\sqrt{s^2}) | (\displaystyle s=\sqrt{\frac{1}{n-1}\sum (x_i-\bar{x})^2}) |
| Coefficient of variation (CV) | Standardised spread | (\displaystyle CV=\frac{s}{\bar{x}}\times100%) |
| Skewness (\gamma_1) | Asymmetry | (\displaystyle \gamma_1=\frac{\frac{1}{n}\sum (x_i-\bar{x})^3}{s^3}) |
| Kurtosis (\gamma_2) | Tail heaviness | (\displaystyle \gamma_2=\frac{\frac{1}{n}\sum (x_i-\bar{x})^4}{s^4}-3) |
Tricky points
- Use (n-1) in variance to get an unbiased estimator.
- Skewness > 0 → right‑skewed; < 0 → left‑skewed.
- Excess kurtosis > 0 → leptokurtic (heavy tails).
3. Probability Distributions
| Distribution | PMF / PDF | Key Parameters | When to use |
|---|---|---|---|
| Binomial | (\displaystyle P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}) | (n) trials, success prob (p) | Discrete “success/failure” |
| Poisson | (\displaystyle P(X=k)=\frac{e^{-\lambda}\lambda^k}{k!}) | Mean = variance = (\lambda) | Rare events per unit time/space |
| Normal | (\displaystyle f(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}) | Mean (\mu), SD (\sigma) | Continuous, CLT |
| Exponential | (\displaystyle f(x)=\lambda e^{-\lambda x}) | Rate (\lambda) | Waiting time between Poisson events |
| Gamma | (\displaystyle f(x)=\frac{\lambda^k}{\Gamma(k)}x^{k-1}e^{-\lambda x}) | Shape (k), rate (\lambda) | Sum of exponentials |
| t‑distribution | (\displaystyle f(t)=\frac{\Gamma(\frac{v+1}{2})}{\sqrt{v\pi},\Gamma(\frac{v}{2})}\Big(1+\frac{t^2}{v}\Big)^{-\frac{v+1}{2}}) | df (v) | Small‑sample mean inference |
| χ² | (\displaystyle f(x)=\frac{1}{2^{v/2}\Gamma(v/2)}x^{v/2-1}e^{-x/2}) | df (v) | Goodness‑of‑fit, variance tests |
| F | (\displaystyle f(F)=\frac{\Gamma(\frac{v_1+v_2}{2})}{\Gamma(\frac{v_1}{2})\Gamma(\frac{v_2}{2})}\Big(\frac{v_1}{v_2}\Big)^{v_1/2}F^{v_1/2-1}\Big(1+\frac{v_1}{v_2}F\Big)^{-(v_1+v_2)/2}) | df1 (v_1), df2 (v_2) | Ratio of variances |
| Beta | (\displaystyle f(x)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1}(1-x)^{\beta-1}) | (\alpha |