How to use this document. Run the code along with
me. Comments start with #. Today we practice Problem Set
2–style tasks using the gapminder dataset, and we compare
dplyr and base R approaches side by
side.
Today’s plan
- Setup: clear environment, load packages, import dataset
- Exploring the data
- Summary statistics & filtering
- Correlation and regression
- Visualization
Setup: Environment
& Packages
rm(list = ls()) # clean environment
# Load the packages
library(tidyverse)
library(ggplot2)
library(gapminder)
In the live session we set the working directory with
setwd(...). In this document the data file
gapminder.csv sits next to the .Rmd, so no
setwd() is needed.
gapminder <- read.csv("gapminder.csv")
# In RStudio you would run View(gapminder); here we preview the first rows.
knitr::kable(head(gapminder), caption = "First rows of the gapminder data")
First rows of the gapminder data
| Afghanistan |
Asia |
1952 |
28.801 |
8425333 |
779.4453 |
| Afghanistan |
Asia |
1957 |
30.332 |
9240934 |
820.8530 |
| Afghanistan |
Asia |
1962 |
31.997 |
10267083 |
853.1007 |
| Afghanistan |
Asia |
1967 |
34.020 |
11537966 |
836.1971 |
| Afghanistan |
Asia |
1972 |
36.088 |
13079460 |
739.9811 |
| Afghanistan |
Asia |
1977 |
38.438 |
14880372 |
786.1134 |
#> country continent year lifeExp
#> Length:1704 Length:1704 Min. :1952 Min. :23.60
#> Class :character Class :character 1st Qu.:1966 1st Qu.:48.20
#> Mode :character Mode :character Median :1980 Median :60.71
#> Mean :1980 Mean :59.47
#> 3rd Qu.:1993 3rd Qu.:70.85
#> Max. :2007 Max. :82.60
#> pop gdpPercap
#> Min. :6.001e+04 Min. : 241.2
#> 1st Qu.:2.794e+06 1st Qu.: 1202.1
#> Median :7.024e+06 Median : 3531.8
#> Mean :2.960e+07 Mean : 7215.3
#> 3rd Qu.:1.959e+07 3rd Qu.: 9325.5
#> Max. :1.319e+09 Max. :113523.1
Simple codebook
country |
name of the country (categorical) |
continent |
continent of the country (categorical) |
year |
year of observation, 1952–2007 (numeric) |
lifeExp |
life expectancy at birth in years (numeric) |
pop |
total population (numeric) |
gdpPercap |
GDP per capita (numeric) |
Exploring the Data
(a) How many rows (observations) and columns (variables) are
in the dataset?
#> [1] 1704
#> [1] 6
(b) How many unique countries are in the country
variable?
# Base R way
length(unique(gapminder$country))
#> [1] 142
# dplyr way
n_distinct(gapminder$country)
#> [1] 142
(c) Count the number of observations for each
continent. With dplyr use
count(); with base R use
table().
# dplyr way
gapminder %>% count(continent)
# Base R way
table(gapminder$continent)
#>
#> Africa Americas Asia Europe Oceania
#> 624 300 396 360 24
Summary Statistics and
Filtering
(a) In 2007, how many countries had a life expectancy
(lifeExp) at least 80?
gapminder %>% filter(year == 2007, lifeExp >= 80) %>% count()
(b) What is the median life expectancy in Asia in
2007? Remember: if a variable is a character type, wrap the
value in quotes ("Asia").
# dplyr way
asia_med <- gapminder %>%
filter(year == 2007, continent == "Asia") %>%
summarize(median = median(lifeExp, na.rm = TRUE))
asia_med
# base R way
median(gapminder$lifeExp[gapminder$year == 2007 & gapminder$continent == "Asia"],
na.rm = TRUE)
#> [1] 72.396
(c) What is the difference in median life expectancy between
Asia and Africa in 2007?
africa_med <- gapminder %>%
filter(year == 2007, continent == "Africa") %>%
summarize(median = median(lifeExp, na.rm = TRUE))
asia_med$median - africa_med$median
#> [1] 19.4695
Another way, using subset() in base R:
asia <- subset(gapminder, year == 2007 & continent == "Asia")
africa <- subset(gapminder, year == 2007 & continent == "Africa")
median_asia <- median(asia$lifeExp, na.rm = TRUE)
median_africa <- median(africa$lifeExp, na.rm = TRUE)
median_asia - median_africa
#> [1] 19.4695
Correlation and
Regression
(a) Using 2007 data, compute the correlation between GDP per
capita (gdpPercap) and life expectancy
(lifeExp). First create a dataset with only the
2007 observations, then calculate the correlation.
gap07 <- gapminder %>% filter(year == 2007)
cor(gap07$gdpPercap, gap07$lifeExp, use = "complete.obs")
#> [1] 0.6786624
(b) Run a simple linear regression with
lifeExp ~ gdpPercap.
m1 <- lm(lifeExp ~ gdpPercap, data = gap07)
summary(m1)
#>
#> Call:
#> lm(formula = lifeExp ~ gdpPercap, data = gap07)
#>
#> Residuals:
#> Min 1Q Median 3Q Max
#> -22.828 -6.316 1.922 6.898 13.128
#>
#> Coefficients:
#> Estimate Std. Error t value Pr(>|t|)
#> (Intercept) 5.957e+01 1.010e+00 58.95 <2e-16 ***
#> gdpPercap 6.371e-04 5.827e-05 10.93 <2e-16 ***
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 8.899 on 140 degrees of freedom
#> Multiple R-squared: 0.4606, Adjusted R-squared: 0.4567
#> F-statistic: 119.5 on 1 and 140 DF, p-value: < 2.2e-16
(c) Interpret the regression coefficients (slope and
intercept).
- Intercept ≈ 59.6. When GDP per capita = 0, the
model predicts a life expectancy of about 59.6 years.
- Slope ≈ 0.00064. For every additional $1 of GDP per
capita, life expectancy increases by about 0.00064 years.
- Both coefficients have p-values < 0.001
(
***), so GDP per capita is a highly significant predictor
of life expectancy in 2007.
- R-squared ≈ 0.46. About 46% of the variation in
life expectancy across countries in 2007 is explained by GDP per capita
alone.
Visualization
Create a scatterplot of gdpPercap and
lifeExp for 2007, and add a regression line.
ggplot(gap07, aes(x = gdpPercap, y = lifeExp)) +
geom_point(color = "steelblue") +
labs(
x = "GDP per Capita",
y = "Life Expectancy (years)",
title = "GDP per Capita vs Life Expectancy (2007)"
) +
geom_smooth(method = "lm", se = FALSE, color = "red", size = 1.2)

Wrap-up
Key takeaways
dplyr verbs (filter, count,
summarize, %>%) and base R
(subset, table, [ ]) often give
the same answer — pick whichever is clearer.
- Use
n_distinct() (or length(unique())) to
count distinct categories.
cor(x, y, use = "complete.obs") handles missing values
when measuring association.
- Fit a linear model with
lm(y ~ x, data = ...) and read
it with summary().
- Layer a regression line onto a scatterplot with
geom_smooth(method = "lm").
LS0tCnRpdGxlOiAiTGFiIDUg4oCUIFByYWN0aWNlIHdpdGggdGhlIGdhcG1pbmRlciBEYXRhIChQcm9ibGVtIFNldCAyKSIKc3VidGl0bGU6ICJRdWFudGl0YXRpdmUgUmVhc29uaW5nIMK3IExBQiA0MTMiCmF1dGhvcjogIkluc3RydWN0b3I6IFN1YmluIE5hIgpkYXRlOiAiT2N0b2JlciAxLCAyMDI1IgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRoZW1lOiBmbGF0bHkKICAgIGhpZ2hsaWdodDogdGFuZ28KICAgIHRvYzogdHJ1ZQogICAgdG9jX2Zsb2F0OiB0cnVlCiAgICB0b2NfZGVwdGg6IDIKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBlY2hvID0gVFJVRSwgbWVzc2FnZSA9IEZBTFNFLCB3YXJuaW5nID0gRkFMU0UsCiAgZmlnLmFsaWduID0gImNlbnRlciIsIGZpZy53aWR0aCA9IDcsIGZpZy5oZWlnaHQgPSA0LjIsCiAgY29tbWVudCA9ICIjPiIKKQpgYGAKCj4gKipIb3cgdG8gdXNlIHRoaXMgZG9jdW1lbnQuKiogUnVuIHRoZSBjb2RlIGFsb25nIHdpdGggbWUuIENvbW1lbnRzIHN0YXJ0IHdpdGggYCNgLgo+IFRvZGF5IHdlIHByYWN0aWNlIFByb2JsZW0gU2V0IDLigJNzdHlsZSB0YXNrcyB1c2luZyB0aGUgYGdhcG1pbmRlcmAgZGF0YXNldCwgYW5kCj4gd2UgY29tcGFyZSAqKmRwbHlyKiogYW5kICoqYmFzZSBSKiogYXBwcm9hY2hlcyBzaWRlIGJ5IHNpZGUuCgoqKlRvZGF5J3MgcGxhbioqCgowLiBTZXR1cDogY2xlYXIgZW52aXJvbm1lbnQsIGxvYWQgcGFja2FnZXMsIGltcG9ydCBkYXRhc2V0CjEuIEV4cGxvcmluZyB0aGUgZGF0YQoyLiBTdW1tYXJ5IHN0YXRpc3RpY3MgJiBmaWx0ZXJpbmcKMy4gQ29ycmVsYXRpb24gYW5kIHJlZ3Jlc3Npb24KNC4gVmlzdWFsaXphdGlvbgoKLS0tCgojIFNldHVwOiBFbnZpcm9ubWVudCAmIFBhY2thZ2VzCgpgYGB7ciBjbGVhbi1zZXR1cH0Kcm0obGlzdCA9IGxzKCkpICAgIyBjbGVhbiBlbnZpcm9ubWVudAoKIyBMb2FkIHRoZSBwYWNrYWdlcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KGdhcG1pbmRlcikKYGBgCgo+IEluIHRoZSBsaXZlIHNlc3Npb24gd2Ugc2V0IHRoZSB3b3JraW5nIGRpcmVjdG9yeSB3aXRoIGBzZXR3ZCguLi4pYC4gSW4gdGhpcwo+IGRvY3VtZW50IHRoZSBkYXRhIGZpbGUgYGdhcG1pbmRlci5jc3ZgIHNpdHMgbmV4dCB0byB0aGUgYC5SbWRgLCBzbyBubwo+IGBzZXR3ZCgpYCBpcyBuZWVkZWQuCgpgYGB7ciBpbXBvcnR9CmdhcG1pbmRlciA8LSByZWFkLmNzdigiZ2FwbWluZGVyLmNzdiIpCgojIEluIFJTdHVkaW8geW91IHdvdWxkIHJ1biBWaWV3KGdhcG1pbmRlcik7IGhlcmUgd2UgcHJldmlldyB0aGUgZmlyc3Qgcm93cy4Ka25pdHI6OmthYmxlKGhlYWQoZ2FwbWluZGVyKSwgY2FwdGlvbiA9ICJGaXJzdCByb3dzIG9mIHRoZSBnYXBtaW5kZXIgZGF0YSIpCmBgYAoKYGBge3Igc3VtbWFyeX0Kc3VtbWFyeShnYXBtaW5kZXIpCmBgYAoKKipTaW1wbGUgY29kZWJvb2sqKgoKfCBWYXJpYWJsZSB8IERlc2NyaXB0aW9uIHwKfC0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwKfCBgY291bnRyeWAgfCBuYW1lIG9mIHRoZSBjb3VudHJ5IChjYXRlZ29yaWNhbCkgfAp8IGBjb250aW5lbnRgIHwgY29udGluZW50IG9mIHRoZSBjb3VudHJ5IChjYXRlZ29yaWNhbCkgfAp8IGB5ZWFyYCB8IHllYXIgb2Ygb2JzZXJ2YXRpb24sIDE5NTLigJMyMDA3IChudW1lcmljKSB8CnwgYGxpZmVFeHBgIHwgbGlmZSBleHBlY3RhbmN5IGF0IGJpcnRoIGluIHllYXJzIChudW1lcmljKSB8CnwgYHBvcGAgfCB0b3RhbCBwb3B1bGF0aW9uIChudW1lcmljKSB8CnwgYGdkcFBlcmNhcGAgfCBHRFAgcGVyIGNhcGl0YSAobnVtZXJpYykgfAoKIyBFeHBsb3JpbmcgdGhlIERhdGEKCioqKGEpIEhvdyBtYW55IHJvd3MgKG9ic2VydmF0aW9ucykgYW5kIGNvbHVtbnMgKHZhcmlhYmxlcykgYXJlIGluIHRoZSBkYXRhc2V0PyoqCgpgYGB7ciBxMWF9Cm5yb3coZ2FwbWluZGVyKQpuY29sKGdhcG1pbmRlcikKYGBgCgoqKihiKSBIb3cgbWFueSB1bmlxdWUgY291bnRyaWVzIGFyZSBpbiB0aGUgYGNvdW50cnlgIHZhcmlhYmxlPyoqCgpgYGB7ciBxMWJ9CiMgQmFzZSBSIHdheQpsZW5ndGgodW5pcXVlKGdhcG1pbmRlciRjb3VudHJ5KSkKCiMgZHBseXIgd2F5Cm5fZGlzdGluY3QoZ2FwbWluZGVyJGNvdW50cnkpCmBgYAoKKiooYykgQ291bnQgdGhlIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgZm9yIGVhY2ggY29udGluZW50LioqCldpdGggKipkcGx5cioqIHVzZSBgY291bnQoKWA7IHdpdGggKipiYXNlIFIqKiB1c2UgYHRhYmxlKClgLgoKYGBge3IgcTFjfQojIGRwbHlyIHdheQpnYXBtaW5kZXIgJT4lIGNvdW50KGNvbnRpbmVudCkKCiMgQmFzZSBSIHdheQp0YWJsZShnYXBtaW5kZXIkY29udGluZW50KQpgYGAKCiMgU3VtbWFyeSBTdGF0aXN0aWNzIGFuZCBGaWx0ZXJpbmcKCioqKGEpIEluIDIwMDcsIGhvdyBtYW55IGNvdW50cmllcyBoYWQgYSBsaWZlIGV4cGVjdGFuY3kgKGBsaWZlRXhwYCkgYXQgbGVhc3QgODA/KioKCmBgYHtyIHEyYX0KZ2FwbWluZGVyICU+JSBmaWx0ZXIoeWVhciA9PSAyMDA3LCBsaWZlRXhwID49IDgwKSAlPiUgY291bnQoKQpgYGAKCioqKGIpIFdoYXQgaXMgdGhlIG1lZGlhbiBsaWZlIGV4cGVjdGFuY3kgaW4gQXNpYSBpbiAyMDA3PyoqClJlbWVtYmVyOiBpZiBhIHZhcmlhYmxlIGlzIGEgY2hhcmFjdGVyIHR5cGUsIHdyYXAgdGhlIHZhbHVlIGluIHF1b3RlcyAoYCJBc2lhImApLgoKYGBge3IgcTJifQojIGRwbHlyIHdheQphc2lhX21lZCA8LSBnYXBtaW5kZXIgJT4lCiAgZmlsdGVyKHllYXIgPT0gMjAwNywgY29udGluZW50ID09ICJBc2lhIikgJT4lCiAgc3VtbWFyaXplKG1lZGlhbiA9IG1lZGlhbihsaWZlRXhwLCBuYS5ybSA9IFRSVUUpKQphc2lhX21lZAoKIyBiYXNlIFIgd2F5Cm1lZGlhbihnYXBtaW5kZXIkbGlmZUV4cFtnYXBtaW5kZXIkeWVhciA9PSAyMDA3ICYgZ2FwbWluZGVyJGNvbnRpbmVudCA9PSAiQXNpYSJdLAogICAgICAgbmEucm0gPSBUUlVFKQpgYGAKCioqKGMpIFdoYXQgaXMgdGhlIGRpZmZlcmVuY2UgaW4gbWVkaWFuIGxpZmUgZXhwZWN0YW5jeSBiZXR3ZWVuIEFzaWEgYW5kIEFmcmljYSBpbiAyMDA3PyoqCgpgYGB7ciBxMmN9CmFmcmljYV9tZWQgPC0gZ2FwbWluZGVyICU+JQogIGZpbHRlcih5ZWFyID09IDIwMDcsIGNvbnRpbmVudCA9PSAiQWZyaWNhIikgJT4lCiAgc3VtbWFyaXplKG1lZGlhbiA9IG1lZGlhbihsaWZlRXhwLCBuYS5ybSA9IFRSVUUpKQoKYXNpYV9tZWQkbWVkaWFuIC0gYWZyaWNhX21lZCRtZWRpYW4KYGBgCgpBbm90aGVyIHdheSwgdXNpbmcgYHN1YnNldCgpYCBpbiBiYXNlIFI6CgpgYGB7ciBxMmMtYmFzZVJ9CmFzaWEgICA8LSBzdWJzZXQoZ2FwbWluZGVyLCB5ZWFyID09IDIwMDcgJiBjb250aW5lbnQgPT0gIkFzaWEiKQphZnJpY2EgPC0gc3Vic2V0KGdhcG1pbmRlciwgeWVhciA9PSAyMDA3ICYgY29udGluZW50ID09ICJBZnJpY2EiKQoKbWVkaWFuX2FzaWEgICA8LSBtZWRpYW4oYXNpYSRsaWZlRXhwLCBuYS5ybSA9IFRSVUUpCm1lZGlhbl9hZnJpY2EgPC0gbWVkaWFuKGFmcmljYSRsaWZlRXhwLCBuYS5ybSA9IFRSVUUpCgptZWRpYW5fYXNpYSAtIG1lZGlhbl9hZnJpY2EKYGBgCgojIENvcnJlbGF0aW9uIGFuZCBSZWdyZXNzaW9uCgoqKihhKSBVc2luZyAyMDA3IGRhdGEsIGNvbXB1dGUgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gR0RQIHBlciBjYXBpdGEgKGBnZHBQZXJjYXBgKSBhbmQgbGlmZSBleHBlY3RhbmN5IChgbGlmZUV4cGApLioqCkZpcnN0IGNyZWF0ZSBhIGRhdGFzZXQgd2l0aCBvbmx5IHRoZSAyMDA3IG9ic2VydmF0aW9ucywgdGhlbiBjYWxjdWxhdGUgdGhlIGNvcnJlbGF0aW9uLgoKYGBge3IgcTNhfQpnYXAwNyA8LSBnYXBtaW5kZXIgJT4lIGZpbHRlcih5ZWFyID09IDIwMDcpCmNvcihnYXAwNyRnZHBQZXJjYXAsIGdhcDA3JGxpZmVFeHAsIHVzZSA9ICJjb21wbGV0ZS5vYnMiKQpgYGAKCioqKGIpIFJ1biBhIHNpbXBsZSBsaW5lYXIgcmVncmVzc2lvbiB3aXRoIGBsaWZlRXhwIH4gZ2RwUGVyY2FwYC4qKgoKYGBge3IgcTNifQptMSA8LSBsbShsaWZlRXhwIH4gZ2RwUGVyY2FwLCBkYXRhID0gZ2FwMDcpCnN1bW1hcnkobTEpCmBgYAoKKiooYykgSW50ZXJwcmV0IHRoZSByZWdyZXNzaW9uIGNvZWZmaWNpZW50cyAoc2xvcGUgYW5kIGludGVyY2VwdCkuKioKCi0gKipJbnRlcmNlcHQg4omIIDU5LjYuKiogV2hlbiBHRFAgcGVyIGNhcGl0YSA9IDAsIHRoZSBtb2RlbCBwcmVkaWN0cyBhIGxpZmUKICBleHBlY3RhbmN5IG9mIGFib3V0IDU5LjYgeWVhcnMuCi0gKipTbG9wZSDiiYggMC4wMDA2NC4qKiBGb3IgZXZlcnkgYWRkaXRpb25hbCBcJDEgb2YgR0RQIHBlciBjYXBpdGEsIGxpZmUKICBleHBlY3RhbmN5IGluY3JlYXNlcyBieSBhYm91dCAwLjAwMDY0IHllYXJzLgotIEJvdGggY29lZmZpY2llbnRzIGhhdmUgKnAqLXZhbHVlcyA8IDAuMDAxIChgKioqYCksIHNvIEdEUCBwZXIgY2FwaXRhIGlzIGEKICBoaWdobHkgc2lnbmlmaWNhbnQgcHJlZGljdG9yIG9mIGxpZmUgZXhwZWN0YW5jeSBpbiAyMDA3LgotICoqUi1zcXVhcmVkIOKJiCAwLjQ2LioqIEFib3V0IDQ2JSBvZiB0aGUgdmFyaWF0aW9uIGluIGxpZmUgZXhwZWN0YW5jeSBhY3Jvc3MKICBjb3VudHJpZXMgaW4gMjAwNyBpcyBleHBsYWluZWQgYnkgR0RQIHBlciBjYXBpdGEgYWxvbmUuCgojIFZpc3VhbGl6YXRpb24KCkNyZWF0ZSBhIHNjYXR0ZXJwbG90IG9mIGBnZHBQZXJjYXBgIGFuZCBgbGlmZUV4cGAgZm9yIDIwMDcsIGFuZCBhZGQgYSByZWdyZXNzaW9uIGxpbmUuCgpgYGB7ciBxNC1wbG90fQpnZ3Bsb3QoZ2FwMDcsIGFlcyh4ID0gZ2RwUGVyY2FwLCB5ID0gbGlmZUV4cCkpICsKICBnZW9tX3BvaW50KGNvbG9yID0gInN0ZWVsYmx1ZSIpICsKICBsYWJzKAogICAgeCA9ICJHRFAgcGVyIENhcGl0YSIsCiAgICB5ID0gIkxpZmUgRXhwZWN0YW5jeSAoeWVhcnMpIiwKICAgIHRpdGxlID0gIkdEUCBwZXIgQ2FwaXRhIHZzIExpZmUgRXhwZWN0YW5jeSAoMjAwNykiCiAgKSArCiAgZ2VvbV9zbW9vdGgobWV0aG9kID0gImxtIiwgc2UgPSBGQUxTRSwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEuMikKYGBgCgojIFdyYXAtdXAKCioqS2V5IHRha2Vhd2F5cyoqCgotIGBkcGx5cmAgdmVyYnMgKGBmaWx0ZXJgLCBgY291bnRgLCBgc3VtbWFyaXplYCwgYCU+JWApIGFuZCBiYXNlIFIgKGBzdWJzZXRgLAogIGB0YWJsZWAsIGBbIF1gKSBvZnRlbiBnaXZlIHRoZSBzYW1lIGFuc3dlciDigJQgcGljayB3aGljaGV2ZXIgaXMgY2xlYXJlci4KLSBVc2UgYG5fZGlzdGluY3QoKWAgKG9yIGBsZW5ndGgodW5pcXVlKCkpYCkgdG8gY291bnQgZGlzdGluY3QgY2F0ZWdvcmllcy4KLSBgY29yKHgsIHksIHVzZSA9ICJjb21wbGV0ZS5vYnMiKWAgaGFuZGxlcyBtaXNzaW5nIHZhbHVlcyB3aGVuIG1lYXN1cmluZyBhc3NvY2lhdGlvbi4KLSBGaXQgYSBsaW5lYXIgbW9kZWwgd2l0aCBgbG0oeSB+IHgsIGRhdGEgPSAuLi4pYCBhbmQgcmVhZCBpdCB3aXRoIGBzdW1tYXJ5KClgLgotIExheWVyIGEgcmVncmVzc2lvbiBsaW5lIG9udG8gYSBzY2F0dGVycGxvdCB3aXRoIGBnZW9tX3Ntb290aChtZXRob2QgPSAibG0iKWAuCg==