How to use this document. Run the code along with me. Comments start with #. Today we practice Problem Set 2–style tasks using the gapminder dataset, and we compare dplyr and base R approaches side by side.

Today’s plan

  1. Setup: clear environment, load packages, import dataset
  2. Exploring the data
  3. Summary statistics & filtering
  4. Correlation and regression
  5. Visualization

1 Setup: Environment & Packages

rm(list = ls())   # clean environment

# Load the packages
library(tidyverse)
library(ggplot2)
library(gapminder)

In the live session we set the working directory with setwd(...). In this document the data file gapminder.csv sits next to the .Rmd, so no setwd() is needed.

gapminder <- read.csv("gapminder.csv")

# In RStudio you would run View(gapminder); here we preview the first rows.
knitr::kable(head(gapminder), caption = "First rows of the gapminder data")
First rows of the gapminder data
country continent year lifeExp pop gdpPercap
Afghanistan Asia 1952 28.801 8425333 779.4453
Afghanistan Asia 1957 30.332 9240934 820.8530
Afghanistan Asia 1962 31.997 10267083 853.1007
Afghanistan Asia 1967 34.020 11537966 836.1971
Afghanistan Asia 1972 36.088 13079460 739.9811
Afghanistan Asia 1977 38.438 14880372 786.1134
summary(gapminder)
#>    country           continent              year         lifeExp     
#>  Length:1704        Length:1704        Min.   :1952   Min.   :23.60  
#>  Class :character   Class :character   1st Qu.:1966   1st Qu.:48.20  
#>  Mode  :character   Mode  :character   Median :1980   Median :60.71  
#>                                        Mean   :1980   Mean   :59.47  
#>                                        3rd Qu.:1993   3rd Qu.:70.85  
#>                                        Max.   :2007   Max.   :82.60  
#>       pop              gdpPercap       
#>  Min.   :6.001e+04   Min.   :   241.2  
#>  1st Qu.:2.794e+06   1st Qu.:  1202.1  
#>  Median :7.024e+06   Median :  3531.8  
#>  Mean   :2.960e+07   Mean   :  7215.3  
#>  3rd Qu.:1.959e+07   3rd Qu.:  9325.5  
#>  Max.   :1.319e+09   Max.   :113523.1

Simple codebook

Variable Description
country name of the country (categorical)
continent continent of the country (categorical)
year year of observation, 1952–2007 (numeric)
lifeExp life expectancy at birth in years (numeric)
pop total population (numeric)
gdpPercap GDP per capita (numeric)

2 Exploring the Data

(a) How many rows (observations) and columns (variables) are in the dataset?

nrow(gapminder)
#> [1] 1704
ncol(gapminder)
#> [1] 6

(b) How many unique countries are in the country variable?

# Base R way
length(unique(gapminder$country))
#> [1] 142
# dplyr way
n_distinct(gapminder$country)
#> [1] 142

(c) Count the number of observations for each continent. With dplyr use count(); with base R use table().

# dplyr way
gapminder %>% count(continent)
# Base R way
table(gapminder$continent)
#> 
#>   Africa Americas     Asia   Europe  Oceania 
#>      624      300      396      360       24

3 Summary Statistics and Filtering

(a) In 2007, how many countries had a life expectancy (lifeExp) at least 80?

gapminder %>% filter(year == 2007, lifeExp >= 80) %>% count()

(b) What is the median life expectancy in Asia in 2007? Remember: if a variable is a character type, wrap the value in quotes ("Asia").

# dplyr way
asia_med <- gapminder %>%
  filter(year == 2007, continent == "Asia") %>%
  summarize(median = median(lifeExp, na.rm = TRUE))
asia_med
# base R way
median(gapminder$lifeExp[gapminder$year == 2007 & gapminder$continent == "Asia"],
       na.rm = TRUE)
#> [1] 72.396

(c) What is the difference in median life expectancy between Asia and Africa in 2007?

africa_med <- gapminder %>%
  filter(year == 2007, continent == "Africa") %>%
  summarize(median = median(lifeExp, na.rm = TRUE))

asia_med$median - africa_med$median
#> [1] 19.4695

Another way, using subset() in base R:

asia   <- subset(gapminder, year == 2007 & continent == "Asia")
africa <- subset(gapminder, year == 2007 & continent == "Africa")

median_asia   <- median(asia$lifeExp, na.rm = TRUE)
median_africa <- median(africa$lifeExp, na.rm = TRUE)

median_asia - median_africa
#> [1] 19.4695

4 Correlation and Regression

(a) Using 2007 data, compute the correlation between GDP per capita (gdpPercap) and life expectancy (lifeExp). First create a dataset with only the 2007 observations, then calculate the correlation.

gap07 <- gapminder %>% filter(year == 2007)
cor(gap07$gdpPercap, gap07$lifeExp, use = "complete.obs")
#> [1] 0.6786624

(b) Run a simple linear regression with lifeExp ~ gdpPercap.

m1 <- lm(lifeExp ~ gdpPercap, data = gap07)
summary(m1)
#> 
#> Call:
#> lm(formula = lifeExp ~ gdpPercap, data = gap07)
#> 
#> Residuals:
#>     Min      1Q  Median      3Q     Max 
#> -22.828  -6.316   1.922   6.898  13.128 
#> 
#> Coefficients:
#>              Estimate Std. Error t value Pr(>|t|)    
#> (Intercept) 5.957e+01  1.010e+00   58.95   <2e-16 ***
#> gdpPercap   6.371e-04  5.827e-05   10.93   <2e-16 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> Residual standard error: 8.899 on 140 degrees of freedom
#> Multiple R-squared:  0.4606, Adjusted R-squared:  0.4567 
#> F-statistic: 119.5 on 1 and 140 DF,  p-value: < 2.2e-16

(c) Interpret the regression coefficients (slope and intercept).

  • Intercept ≈ 59.6. When GDP per capita = 0, the model predicts a life expectancy of about 59.6 years.
  • Slope ≈ 0.00064. For every additional $1 of GDP per capita, life expectancy increases by about 0.00064 years.
  • Both coefficients have p-values < 0.001 (***), so GDP per capita is a highly significant predictor of life expectancy in 2007.
  • R-squared ≈ 0.46. About 46% of the variation in life expectancy across countries in 2007 is explained by GDP per capita alone.

5 Visualization

Create a scatterplot of gdpPercap and lifeExp for 2007, and add a regression line.

ggplot(gap07, aes(x = gdpPercap, y = lifeExp)) +
  geom_point(color = "steelblue") +
  labs(
    x = "GDP per Capita",
    y = "Life Expectancy (years)",
    title = "GDP per Capita vs Life Expectancy (2007)"
  ) +
  geom_smooth(method = "lm", se = FALSE, color = "red", size = 1.2)

6 Wrap-up

Key takeaways

  • dplyr verbs (filter, count, summarize, %>%) and base R (subset, table, [ ]) often give the same answer — pick whichever is clearer.
  • Use n_distinct() (or length(unique())) to count distinct categories.
  • cor(x, y, use = "complete.obs") handles missing values when measuring association.
  • Fit a linear model with lm(y ~ x, data = ...) and read it with summary().
  • Layer a regression line onto a scatterplot with geom_smooth(method = "lm").
LS0tCnRpdGxlOiAiTGFiIDUg4oCUIFByYWN0aWNlIHdpdGggdGhlIGdhcG1pbmRlciBEYXRhIChQcm9ibGVtIFNldCAyKSIKc3VidGl0bGU6ICJRdWFudGl0YXRpdmUgUmVhc29uaW5nIMK3IExBQiA0MTMiCmF1dGhvcjogIkluc3RydWN0b3I6IFN1YmluIE5hIgpkYXRlOiAiT2N0b2JlciAxLCAyMDI1IgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRoZW1lOiBmbGF0bHkKICAgIGhpZ2hsaWdodDogdGFuZ28KICAgIHRvYzogdHJ1ZQogICAgdG9jX2Zsb2F0OiB0cnVlCiAgICB0b2NfZGVwdGg6IDIKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBlY2hvID0gVFJVRSwgbWVzc2FnZSA9IEZBTFNFLCB3YXJuaW5nID0gRkFMU0UsCiAgZmlnLmFsaWduID0gImNlbnRlciIsIGZpZy53aWR0aCA9IDcsIGZpZy5oZWlnaHQgPSA0LjIsCiAgY29tbWVudCA9ICIjPiIKKQpgYGAKCj4gKipIb3cgdG8gdXNlIHRoaXMgZG9jdW1lbnQuKiogUnVuIHRoZSBjb2RlIGFsb25nIHdpdGggbWUuIENvbW1lbnRzIHN0YXJ0IHdpdGggYCNgLgo+IFRvZGF5IHdlIHByYWN0aWNlIFByb2JsZW0gU2V0IDLigJNzdHlsZSB0YXNrcyB1c2luZyB0aGUgYGdhcG1pbmRlcmAgZGF0YXNldCwgYW5kCj4gd2UgY29tcGFyZSAqKmRwbHlyKiogYW5kICoqYmFzZSBSKiogYXBwcm9hY2hlcyBzaWRlIGJ5IHNpZGUuCgoqKlRvZGF5J3MgcGxhbioqCgowLiBTZXR1cDogY2xlYXIgZW52aXJvbm1lbnQsIGxvYWQgcGFja2FnZXMsIGltcG9ydCBkYXRhc2V0CjEuIEV4cGxvcmluZyB0aGUgZGF0YQoyLiBTdW1tYXJ5IHN0YXRpc3RpY3MgJiBmaWx0ZXJpbmcKMy4gQ29ycmVsYXRpb24gYW5kIHJlZ3Jlc3Npb24KNC4gVmlzdWFsaXphdGlvbgoKLS0tCgojIFNldHVwOiBFbnZpcm9ubWVudCAmIFBhY2thZ2VzCgpgYGB7ciBjbGVhbi1zZXR1cH0Kcm0obGlzdCA9IGxzKCkpICAgIyBjbGVhbiBlbnZpcm9ubWVudAoKIyBMb2FkIHRoZSBwYWNrYWdlcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KGdhcG1pbmRlcikKYGBgCgo+IEluIHRoZSBsaXZlIHNlc3Npb24gd2Ugc2V0IHRoZSB3b3JraW5nIGRpcmVjdG9yeSB3aXRoIGBzZXR3ZCguLi4pYC4gSW4gdGhpcwo+IGRvY3VtZW50IHRoZSBkYXRhIGZpbGUgYGdhcG1pbmRlci5jc3ZgIHNpdHMgbmV4dCB0byB0aGUgYC5SbWRgLCBzbyBubwo+IGBzZXR3ZCgpYCBpcyBuZWVkZWQuCgpgYGB7ciBpbXBvcnR9CmdhcG1pbmRlciA8LSByZWFkLmNzdigiZ2FwbWluZGVyLmNzdiIpCgojIEluIFJTdHVkaW8geW91IHdvdWxkIHJ1biBWaWV3KGdhcG1pbmRlcik7IGhlcmUgd2UgcHJldmlldyB0aGUgZmlyc3Qgcm93cy4Ka25pdHI6OmthYmxlKGhlYWQoZ2FwbWluZGVyKSwgY2FwdGlvbiA9ICJGaXJzdCByb3dzIG9mIHRoZSBnYXBtaW5kZXIgZGF0YSIpCmBgYAoKYGBge3Igc3VtbWFyeX0Kc3VtbWFyeShnYXBtaW5kZXIpCmBgYAoKKipTaW1wbGUgY29kZWJvb2sqKgoKfCBWYXJpYWJsZSB8IERlc2NyaXB0aW9uIHwKfC0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwKfCBgY291bnRyeWAgfCBuYW1lIG9mIHRoZSBjb3VudHJ5IChjYXRlZ29yaWNhbCkgfAp8IGBjb250aW5lbnRgIHwgY29udGluZW50IG9mIHRoZSBjb3VudHJ5IChjYXRlZ29yaWNhbCkgfAp8IGB5ZWFyYCB8IHllYXIgb2Ygb2JzZXJ2YXRpb24sIDE5NTLigJMyMDA3IChudW1lcmljKSB8CnwgYGxpZmVFeHBgIHwgbGlmZSBleHBlY3RhbmN5IGF0IGJpcnRoIGluIHllYXJzIChudW1lcmljKSB8CnwgYHBvcGAgfCB0b3RhbCBwb3B1bGF0aW9uIChudW1lcmljKSB8CnwgYGdkcFBlcmNhcGAgfCBHRFAgcGVyIGNhcGl0YSAobnVtZXJpYykgfAoKIyBFeHBsb3JpbmcgdGhlIERhdGEKCioqKGEpIEhvdyBtYW55IHJvd3MgKG9ic2VydmF0aW9ucykgYW5kIGNvbHVtbnMgKHZhcmlhYmxlcykgYXJlIGluIHRoZSBkYXRhc2V0PyoqCgpgYGB7ciBxMWF9Cm5yb3coZ2FwbWluZGVyKQpuY29sKGdhcG1pbmRlcikKYGBgCgoqKihiKSBIb3cgbWFueSB1bmlxdWUgY291bnRyaWVzIGFyZSBpbiB0aGUgYGNvdW50cnlgIHZhcmlhYmxlPyoqCgpgYGB7ciBxMWJ9CiMgQmFzZSBSIHdheQpsZW5ndGgodW5pcXVlKGdhcG1pbmRlciRjb3VudHJ5KSkKCiMgZHBseXIgd2F5Cm5fZGlzdGluY3QoZ2FwbWluZGVyJGNvdW50cnkpCmBgYAoKKiooYykgQ291bnQgdGhlIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgZm9yIGVhY2ggY29udGluZW50LioqCldpdGggKipkcGx5cioqIHVzZSBgY291bnQoKWA7IHdpdGggKipiYXNlIFIqKiB1c2UgYHRhYmxlKClgLgoKYGBge3IgcTFjfQojIGRwbHlyIHdheQpnYXBtaW5kZXIgJT4lIGNvdW50KGNvbnRpbmVudCkKCiMgQmFzZSBSIHdheQp0YWJsZShnYXBtaW5kZXIkY29udGluZW50KQpgYGAKCiMgU3VtbWFyeSBTdGF0aXN0aWNzIGFuZCBGaWx0ZXJpbmcKCioqKGEpIEluIDIwMDcsIGhvdyBtYW55IGNvdW50cmllcyBoYWQgYSBsaWZlIGV4cGVjdGFuY3kgKGBsaWZlRXhwYCkgYXQgbGVhc3QgODA/KioKCmBgYHtyIHEyYX0KZ2FwbWluZGVyICU+JSBmaWx0ZXIoeWVhciA9PSAyMDA3LCBsaWZlRXhwID49IDgwKSAlPiUgY291bnQoKQpgYGAKCioqKGIpIFdoYXQgaXMgdGhlIG1lZGlhbiBsaWZlIGV4cGVjdGFuY3kgaW4gQXNpYSBpbiAyMDA3PyoqClJlbWVtYmVyOiBpZiBhIHZhcmlhYmxlIGlzIGEgY2hhcmFjdGVyIHR5cGUsIHdyYXAgdGhlIHZhbHVlIGluIHF1b3RlcyAoYCJBc2lhImApLgoKYGBge3IgcTJifQojIGRwbHlyIHdheQphc2lhX21lZCA8LSBnYXBtaW5kZXIgJT4lCiAgZmlsdGVyKHllYXIgPT0gMjAwNywgY29udGluZW50ID09ICJBc2lhIikgJT4lCiAgc3VtbWFyaXplKG1lZGlhbiA9IG1lZGlhbihsaWZlRXhwLCBuYS5ybSA9IFRSVUUpKQphc2lhX21lZAoKIyBiYXNlIFIgd2F5Cm1lZGlhbihnYXBtaW5kZXIkbGlmZUV4cFtnYXBtaW5kZXIkeWVhciA9PSAyMDA3ICYgZ2FwbWluZGVyJGNvbnRpbmVudCA9PSAiQXNpYSJdLAogICAgICAgbmEucm0gPSBUUlVFKQpgYGAKCioqKGMpIFdoYXQgaXMgdGhlIGRpZmZlcmVuY2UgaW4gbWVkaWFuIGxpZmUgZXhwZWN0YW5jeSBiZXR3ZWVuIEFzaWEgYW5kIEFmcmljYSBpbiAyMDA3PyoqCgpgYGB7ciBxMmN9CmFmcmljYV9tZWQgPC0gZ2FwbWluZGVyICU+JQogIGZpbHRlcih5ZWFyID09IDIwMDcsIGNvbnRpbmVudCA9PSAiQWZyaWNhIikgJT4lCiAgc3VtbWFyaXplKG1lZGlhbiA9IG1lZGlhbihsaWZlRXhwLCBuYS5ybSA9IFRSVUUpKQoKYXNpYV9tZWQkbWVkaWFuIC0gYWZyaWNhX21lZCRtZWRpYW4KYGBgCgpBbm90aGVyIHdheSwgdXNpbmcgYHN1YnNldCgpYCBpbiBiYXNlIFI6CgpgYGB7ciBxMmMtYmFzZVJ9CmFzaWEgICA8LSBzdWJzZXQoZ2FwbWluZGVyLCB5ZWFyID09IDIwMDcgJiBjb250aW5lbnQgPT0gIkFzaWEiKQphZnJpY2EgPC0gc3Vic2V0KGdhcG1pbmRlciwgeWVhciA9PSAyMDA3ICYgY29udGluZW50ID09ICJBZnJpY2EiKQoKbWVkaWFuX2FzaWEgICA8LSBtZWRpYW4oYXNpYSRsaWZlRXhwLCBuYS5ybSA9IFRSVUUpCm1lZGlhbl9hZnJpY2EgPC0gbWVkaWFuKGFmcmljYSRsaWZlRXhwLCBuYS5ybSA9IFRSVUUpCgptZWRpYW5fYXNpYSAtIG1lZGlhbl9hZnJpY2EKYGBgCgojIENvcnJlbGF0aW9uIGFuZCBSZWdyZXNzaW9uCgoqKihhKSBVc2luZyAyMDA3IGRhdGEsIGNvbXB1dGUgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gR0RQIHBlciBjYXBpdGEgKGBnZHBQZXJjYXBgKSBhbmQgbGlmZSBleHBlY3RhbmN5IChgbGlmZUV4cGApLioqCkZpcnN0IGNyZWF0ZSBhIGRhdGFzZXQgd2l0aCBvbmx5IHRoZSAyMDA3IG9ic2VydmF0aW9ucywgdGhlbiBjYWxjdWxhdGUgdGhlIGNvcnJlbGF0aW9uLgoKYGBge3IgcTNhfQpnYXAwNyA8LSBnYXBtaW5kZXIgJT4lIGZpbHRlcih5ZWFyID09IDIwMDcpCmNvcihnYXAwNyRnZHBQZXJjYXAsIGdhcDA3JGxpZmVFeHAsIHVzZSA9ICJjb21wbGV0ZS5vYnMiKQpgYGAKCioqKGIpIFJ1biBhIHNpbXBsZSBsaW5lYXIgcmVncmVzc2lvbiB3aXRoIGBsaWZlRXhwIH4gZ2RwUGVyY2FwYC4qKgoKYGBge3IgcTNifQptMSA8LSBsbShsaWZlRXhwIH4gZ2RwUGVyY2FwLCBkYXRhID0gZ2FwMDcpCnN1bW1hcnkobTEpCmBgYAoKKiooYykgSW50ZXJwcmV0IHRoZSByZWdyZXNzaW9uIGNvZWZmaWNpZW50cyAoc2xvcGUgYW5kIGludGVyY2VwdCkuKioKCi0gKipJbnRlcmNlcHQg4omIIDU5LjYuKiogV2hlbiBHRFAgcGVyIGNhcGl0YSA9IDAsIHRoZSBtb2RlbCBwcmVkaWN0cyBhIGxpZmUKICBleHBlY3RhbmN5IG9mIGFib3V0IDU5LjYgeWVhcnMuCi0gKipTbG9wZSDiiYggMC4wMDA2NC4qKiBGb3IgZXZlcnkgYWRkaXRpb25hbCBcJDEgb2YgR0RQIHBlciBjYXBpdGEsIGxpZmUKICBleHBlY3RhbmN5IGluY3JlYXNlcyBieSBhYm91dCAwLjAwMDY0IHllYXJzLgotIEJvdGggY29lZmZpY2llbnRzIGhhdmUgKnAqLXZhbHVlcyA8IDAuMDAxIChgKioqYCksIHNvIEdEUCBwZXIgY2FwaXRhIGlzIGEKICBoaWdobHkgc2lnbmlmaWNhbnQgcHJlZGljdG9yIG9mIGxpZmUgZXhwZWN0YW5jeSBpbiAyMDA3LgotICoqUi1zcXVhcmVkIOKJiCAwLjQ2LioqIEFib3V0IDQ2JSBvZiB0aGUgdmFyaWF0aW9uIGluIGxpZmUgZXhwZWN0YW5jeSBhY3Jvc3MKICBjb3VudHJpZXMgaW4gMjAwNyBpcyBleHBsYWluZWQgYnkgR0RQIHBlciBjYXBpdGEgYWxvbmUuCgojIFZpc3VhbGl6YXRpb24KCkNyZWF0ZSBhIHNjYXR0ZXJwbG90IG9mIGBnZHBQZXJjYXBgIGFuZCBgbGlmZUV4cGAgZm9yIDIwMDcsIGFuZCBhZGQgYSByZWdyZXNzaW9uIGxpbmUuCgpgYGB7ciBxNC1wbG90fQpnZ3Bsb3QoZ2FwMDcsIGFlcyh4ID0gZ2RwUGVyY2FwLCB5ID0gbGlmZUV4cCkpICsKICBnZW9tX3BvaW50KGNvbG9yID0gInN0ZWVsYmx1ZSIpICsKICBsYWJzKAogICAgeCA9ICJHRFAgcGVyIENhcGl0YSIsCiAgICB5ID0gIkxpZmUgRXhwZWN0YW5jeSAoeWVhcnMpIiwKICAgIHRpdGxlID0gIkdEUCBwZXIgQ2FwaXRhIHZzIExpZmUgRXhwZWN0YW5jeSAoMjAwNykiCiAgKSArCiAgZ2VvbV9zbW9vdGgobWV0aG9kID0gImxtIiwgc2UgPSBGQUxTRSwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEuMikKYGBgCgojIFdyYXAtdXAKCioqS2V5IHRha2Vhd2F5cyoqCgotIGBkcGx5cmAgdmVyYnMgKGBmaWx0ZXJgLCBgY291bnRgLCBgc3VtbWFyaXplYCwgYCU+JWApIGFuZCBiYXNlIFIgKGBzdWJzZXRgLAogIGB0YWJsZWAsIGBbIF1gKSBvZnRlbiBnaXZlIHRoZSBzYW1lIGFuc3dlciDigJQgcGljayB3aGljaGV2ZXIgaXMgY2xlYXJlci4KLSBVc2UgYG5fZGlzdGluY3QoKWAgKG9yIGBsZW5ndGgodW5pcXVlKCkpYCkgdG8gY291bnQgZGlzdGluY3QgY2F0ZWdvcmllcy4KLSBgY29yKHgsIHksIHVzZSA9ICJjb21wbGV0ZS5vYnMiKWAgaGFuZGxlcyBtaXNzaW5nIHZhbHVlcyB3aGVuIG1lYXN1cmluZyBhc3NvY2lhdGlvbi4KLSBGaXQgYSBsaW5lYXIgbW9kZWwgd2l0aCBgbG0oeSB+IHgsIGRhdGEgPSAuLi4pYCBhbmQgcmVhZCBpdCB3aXRoIGBzdW1tYXJ5KClgLgotIExheWVyIGEgcmVncmVzc2lvbiBsaW5lIG9udG8gYSBzY2F0dGVycGxvdCB3aXRoIGBnZW9tX3Ntb290aChtZXRob2QgPSAibG0iKWAuCg==