M.Tech /Core Science

Data analysis

Laboratory & Practicals · free study notes · Updated

Core Science – M.Tech.

Topic: Data Analysis

Data analysis is the systematic process of inspecting, cleaning, transforming, and modeling data to discover useful information, draw conclusions, and support decision‑making.


1. Simple Explanation

Step What Happens Why It Matters
1. Collect Gather raw data from experiments, sensors, or databases. Provides the material for analysis.
2. Clean Remove errors, handle missing values, and standardize formats. Ensures reliability of results.
3. Explore Use descriptive statistics & visualisations to understand distributions, trends, and outliers. Gives intuition and guides further steps.
4. Model Apply statistical or machine‑learning models (regression, clustering, classification). Quantifies relationships or predicts outcomes.
5. Validate Test model performance using cross‑validation, error metrics, or hypothesis tests. Confirms that the model generalises.
6. Interpret Translate numbers into scientific insight, write reports, and make recommendations. Turns data into actionable knowledge.

2. Key Points

Concept Definition Example
Variables Quantities measured or observed. Temperature (continuous), Species (categorical).
Descriptive Stats Mean, median, variance, standard deviation, percentiles. Mean temperature of a dataset = 23.4 °C.
Correlation Statistical association between two variables. Pearson’s r = 0.78 indicates a strong positive relationship.
Regression Predicts a dependent variable from one or more independent variables. Linear regression: (y = \beta_0 + \beta_1x).
Hypothesis Testing Determines if observed patterns are statistically significant. t‑test for mean difference.
Confidence Interval Range that likely contains the true parameter. 95 % CI for mean = (21.8, 25.0).
Outlier Observation far from others. A temperature reading of 120 °C in a dataset of 20–30 °C.
Missing Data Unrecorded values. Use imputation or deletion.
Overfitting Model captures noise instead of signal. Occurs when too many predictors are used.
Cross‑Validation Technique to assess model’s predictive power. 5‑fold CV.

3. Worked Example

Problem:
You have measured the chlorophyll content (µg g⁻¹) of 10 plant leaves and want to see if leaf area (cm²) predicts chlorophyll.

Leaf Area (cm²) Chlorophyll (µg g⁻¹)
1 12 45
2 15 52
3 9 38
4 20 64
5 14 50
6 18 60
7 11 42
8 17 57
9 13 48
10 16 55

Step 1: Exploratory Data Analysis

  • Mean area: ( \bar{x} = 15.1) cm²
  • Mean chlorophyll: ( \bar{y} = 52.4) µg g⁻¹
  • Scatter plot shows a positive trend.

Step 2: Linear Regression

We fit ( y = \beta_0 + \beta_1 x ).

  • Compute slope: [ \beta_1 = \frac{\sum (x_i - \bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2} = \frac{340}{84} \approx 4.05 ]
  • Compute intercept: [ \beta_0 = \bar{y} - \beta_1\bar{x} = 52.4 - 4.05\times15.1 \approx 14.8 ]

Model:
( \hat{y} = 14.8 + 4.05,x )

Step 3: Model Assessment

  • R²: proportion of variance explained.
    ( R^2 = 0.92 ) → 92 % of chlorophyll variance explained by area.

  • Residuals: small and normally distributed → model fits well.

  • Significance: p‑value for ( \beta_1 < 0.001 ) → strong evidence that area predicts chlorophyll.

Step 4: Interpretation

Leaf area explains most of the variation in chlorophyll content. For each additional cm² of leaf area, chlorophyll increases by ~4 µg g⁻¹.


4. Quick Revision Summary

Topic What to Remember
Data collection Keep a clear protocol; document sources.
Cleaning Check for duplicates, outliers, missing values.
Descriptive stats Mean, SD, median, IQR.
Correlation Pearson r, Spearman ρ, correlation matrix.
Regression Equation, coefficients, R², assumptions.
Hypothesis testing Null vs alternative, p‑value, significance level.
Validation Train‑test split, k‑fold CV, overfitting warning.
Interpretation Translate statistics to scientific conclusions.

Tip for M.Tech. students:
Always start with a question. Your analysis should answer that question or refine it. Document every step, as reproducibility is key in high‑level research.

Your note

Page 1

Doubt hai is topic me?

Free AI teacher se poocho — 24/7, step by step.