What's a ZCTA?
For years I assumed that a city consists of a number of zipcodes, and the city's population was simply the sum of the populations of those zipcodes.It turns out this isn't quite right.
Zipcodes are postal delivery routes. They are a collection of lines (streets) and points (delivery boxes) designed purely for mail trucks to drop off letters as efficiently as possible. They do not have official, enclosed borders. Also, zipcode definitions frequently change.
To make geographical analysis possible, the Census Bureau invented ZCTAs (ZIP Code Tabulation Areas) using Census blocks into artificial, enclosed geographic polygons. Each resulting shape is given a 5-digit name that matches the dominant zipcode of the greatest number of addresses inside it.
The U.S. Census Bureau publishes decennial population by city, and also by these smaller ZCTAs, but never by actual USPS zipcodes. (They do publish other statistics by zipcode.)
A ZCTA polygon is unlikely to perfectly match the geographical area of the dominant zipcode. Moreover, a ZCTA may actually cross a municipal boundary. For example, ZCTA 11001 (Floral Park) and 11040 (New Hyde Park) span both Queens (NYC) and Nassau Counties, but most of their area and population are in Nassau County. So counting population in such ZCTAs in full for NYC would overstate NYC's population. Adding the populations of all the NYC ZCTAs is not going to equal the population of NYC.
Distribution of race in a smaller city
I am using the R package tidycensus to pull data from the U.S. Census Bureau API into R. Census data includes many possible variables. Some broad categories are population, median household income, housing units, median rent, education, occupation, etc. For convenience I chose population by race for this project, although I could have chosen something else. You need to request a one-time Census API key from https://api.census.gov/data/key_signup.html .With the above as background, I decided to examine the distribution of race in a smaller city, Fort Myers, Florida. There are 13 inhabited ZCTAs in the greater Fort Myers area, plus some miscellaneous ZCTAs such as for post office boxes. A map of the greater Fort Myers area with its ZCTAs, which extend beyond Fort Myers City, is as follows, where the dashed red line indicates Fort Myers City limits:
| Race (NH = Non-Hispanic) | Percentage |
|---|---|
| White (NH) | 47.51% |
| Hispanic (any race) | 23.61% |
| Black (NH) | 21.87% |
| Other (NH) | 7.01% |
Superimposing the above table as a pie chart onto the map of Fort Myers City gives the following:
What's the value in all this?
What's the value in all this? Your pharmacy, department store, and video streaming service all use information about you to provide personalized recommendations based on what they think you'll buy or watch next. Do our political parties produce customized political messages for different neighborhoods based on what they think are their residents' major political issues? Is race/ethnicity an important (although of course not the only) variable in people's political issues? I think maybe our political parties should think about making greater use of demographic and geographic information when deciding how to communicate with voters.
R code
There is a lot of R code. I was considerably helped by Microsoft Copilot and Google Gemini.
# Race by ZCTA
# MY_KEY <- "xxxxx"
# Sys.setenv(CENSUS_API_KEY = "MY_KEY")
library(tidycensus)
library(tigris)
library(dplyr)
library(tidyr)
library(stringr)
library(sf)
library(ggplot2)
library(scatterpie)
library(gridExtra)
options(tigris_use_cache = TRUE)
# ------------------------------------------------------------
# Variable Definitions
# ------------------------------------------------------------
# 2020 Census P2 (Place-Level)
vars_p2 <- c(
total = "P2_001N",
hispanic = "P2_002N",
white_nh = "P2_005N",
black_nh = "P2_006N",
aian_nh = "P2_007N",
asian_nh = "P2_008N",
nhpi_nh = "P2_009N",
other_nh = "P2_010N",
multi_nh = "P2_011N"
)
# 2020 Census DHC ZCTA Variables
vars_p5 <- c(
total_pop = "P5_001N",
white_nh = "P5_003N",
black_nh = "P5_004N",
aian_nh = "P5_005N",
asian_nh = "P5_006N",
nhpi_nh = "P5_007N",
other_nh = "P5_008N",
multi_nh = "P5_009N",
hispanic = "P5_010N"
)
# ACS Variables (for individual ZCTA analysis)
vars_acs <- c(
total = "B03002_001",
white_nh = "B03002_003",
black_nh = "B03002_004",
aian_nh = "B03002_005",
asian_nh = "B03002_006",
nhpi_nh = "B03002_007",
other_nh = "B03002_008",
multi_nh = "B03002_009",
hispanic = "B03002_012"
)
# ------------------------------------------------------------
# Geography
# ------------------------------------------------------------
places_fl <- places(state = "FL", year = 2020)
fort_myers_shape <- places_fl %>%
filter(NAME == "Fort Myers")
zctas_fl <- zctas(
year = 2020,
cb = TRUE,
starts_with = "339"
)
# ------------------------------------------------------------
# CITY OF FORT MYERS RACE TABLE
# ------------------------------------------------------------
race_eth_city <- get_decennial(
geography = "place",
state = "FL",
variables = vars_p2,
year = 2020,
sumfile = "pl",
output = "wide"
) %>%
filter(NAME == "Fort Myers city, Florida")
race_city <- race_eth_city %>%
transmute(
white_nh,
hispanic,
black_nh,
other =
aian_nh +
asian_nh +
nhpi_nh +
other_nh +
multi_nh
)
race_table_city <- race_city %>%
pivot_longer(
everything(),
names_to = "race",
values_to = "population"
) %>%
mutate(
pct = population / sum(population)
)
print(race_table_city)
# ------------------------------------------------------------
# CITY OF FORT MYERS MAP
# ------------------------------------------------------------
ggplot(fort_myers_shape) +
geom_sf(
fill = "lightblue",
color = "black"
) +
labs(
title = "City of Fort Myers (2020 Census Boundary)"
) +
theme_minimal() +
theme(
plot.title = element_text(face = "bold"),
axis.text = element_blank(),
axis.ticks = element_blank()
)
# ------------------------------------------------------------
# CITY PIE CHART MAP
# ------------------------------------------------------------
ctr <- st_coordinates(
st_centroid(
st_union(fort_myers_shape)
)
)
pie_df_city <- cbind(
data.frame(
x = ctr[1],
y = ctr[2]
),
race_city
)
ggplot() +
geom_sf(
data = fort_myers_shape,
fill = "lightblue",
color = "black"
) +
geom_scatterpie(
data = pie_df_city,
aes(x = x, y = y, r = 0.03),
cols = names(race_city)
) +
scale_fill_manual(
values = c(
white_nh = "lightgray",
hispanic = "orange",
black_nh = "steelblue",
other = "purple"
)
) +
labs(
title = "Fort Myers City Racial Composition",
fill = "Race/Ethnicity"
) +
coord_sf() +
theme(
plot.title = element_text(face = "bold"),
axis.title = element_blank(),
axis.text = element_blank(),
axis.ticks = element_blank(),
legend.title = element_text(face = "bold"),
legend.text = element_text(face = "bold")
)
# ------------------------------------------------------------
# GREATER FORT MYERS ZCTA MAP
# ------------------------------------------------------------
fort_myers_zctas <- zctas_fl %>%
st_filter(fort_myers_shape,
.predicate = st_intersects)
ggplot() +
geom_sf(
data = fort_myers_zctas,
fill = "lightblue",
color = "black",
alpha = 0.6
) +
geom_sf(
data = fort_myers_shape,
fill = NA,
color = "red",
linewidth = 1.8,
linetype = "dashed"
) +
geom_sf_label(
data = fort_myers_zctas,
aes(label = ZCTA5CE20),
size = 3,
fontface = "bold"
) +
labs(
title = "Greater Fort Myers Area by ZCTA",
subtitle = "Dashed red line indicates city limits",
x = "",
y = "") +
theme_minimal() +
theme(
plot.title = element_text(face = "bold"),
plot.subtitle = element_text(face = "bold"),
axis.text = element_blank(),
axis.ticks = element_blank()
)
# ------------------------------------------------------------
# SINGLE ZCTA ANALYSIS (33905)
# ------------------------------------------------------------
race_eth_33905 <- get_acs(
geography = "zcta",
variables = vars_acs,
year = 2020,
output = "wide"
) %>%
filter(GEOID == "33905")
race_33905 <- race_eth_33905 %>%
transmute(
white_nh = white_nhE,
hispanic = hispanicE,
black_nh = black_nhE,
other =
aian_nhE +
asian_nhE +
nhpi_nhE +
other_nhE +
multi_nhE
)
race_table_33905 <- race_33905 %>%
pivot_longer(
everything(),
names_to = "race",
values_to = "population"
) %>%
mutate(
pct = population / sum(population)
)
print(race_table_33905)
# ------------------------------------------------------------
# ZCTA 33905 PIE MAP
# ------------------------------------------------------------
zcta_33905 <- zctas_fl %>%
filter(ZCTA5CE20 == "33905")
coords_33905 <- st_coordinates(
st_centroid(zcta_33905)
)
pie_df_33905 <- race_33905 %>%
mutate(
x = coords_33905[1,1],
y = coords_33905[1,2]
)
ggplot() +
geom_sf(
data = zcta_33905,
fill = "lightblue",
color = "black"
) +
geom_scatterpie(
data = pie_df_33905,
aes(x = x, y = y, r = 0.025),
cols = c(
"white_nh",
"hispanic",
"black_nh",
"other"
)
) +
scale_fill_manual(
values = c(
white_nh = "lightgray",
hispanic = "orange",
black_nh = "steelblue",
other = "purple"
)
) +
labs(
title = "ZCTA 33905 Racial Composition",
fill = "Race/Ethnicity"
) +
coord_sf() +
theme_minimal() +
theme(
plot.title = element_text(face = "bold"),
axis.title = element_blank(),
axis.text = element_blank(),
axis.ticks = element_blank()
)
# ------------------------------------------------------------
# ALL FORT MYERS AREA ZCTAS
# ------------------------------------------------------------
fort_myers_list <- c(
"33901","33903","33905","33907",
"33908","33912","33913","33916",
"33917","33919","33931","33965",
"33966","33967"
)
fort_myers_master <- get_decennial(
geography = "zcta",
variables = vars_p5,
year = 2020,
sumfile = "dhc",
output = "wide",
geometry = TRUE
)
fort_myers_projected <- fort_myers_master %>%
filter(GEOID %in% fort_myers_list) %>%
mutate(
zcta = str_extract(NAME, "\\d{5}"),
pct_hispanic =
100 * hispanic / total_pop,
pct_black_nh =
100 * black_nh / total_pop,
minority_other_nh =
aian_nh +
asian_nh +
nhpi_nh +
other_nh +
multi_nh
) %>%
st_transform(2237)
# ------------------------------------------------------------
# POPULATION MATRIX
# ------------------------------------------------------------
fort_myers_population_table <- fort_myers_projected %>%
st_drop_geometry() %>%
select(
zcta,
total_pop,
white_nh,
hispanic,
black_nh,
minority_other_nh
) %>%
arrange(zcta)
print(fort_myers_population_table)
# ------------------------------------------------------------
# CENTROIDS
# ------------------------------------------------------------
centroids_master <- fort_myers_projected %>%
st_centroid() %>%
mutate(
X = st_coordinates(.)[,1],
Y = st_coordinates(.)[,2]
) %>%
st_drop_geometry()
top_5_hispanic <- centroids_master %>%
slice_max(
pct_hispanic,
n = 5
) %>%
mutate(
label_text =
paste0(
zcta,
"\n",
round(pct_hispanic, 1),
"%"
)
)
top_5_black <- centroids_master %>%
slice_max(
pct_black_nh,
n = 5
) %>%
mutate(
label_text =
paste0(
zcta,
"\n",
round(pct_black_nh, 1),
"%"
)
)
# ------------------------------------------------------------
# BASE MAP
# ------------------------------------------------------------
base_map <- ggplot() +
geom_sf(
data = fort_myers_projected,
fill = "lightblue",
color = "gray60"
) +
theme_minimal() +
theme(
plot.title = element_text(
face = "bold",
hjust = .5
),
axis.title = element_blank(),
axis.text = element_blank(),
panel.grid = element_blank()
) +
coord_sf(datum = NA)
# ------------------------------------------------------------
# TOP 5 HISPANIC
# ------------------------------------------------------------
map_hispanic <- base_map +
geom_text(
data = top_5_hispanic,
aes(
X,
Y,
label = label_text
),
fontface = "bold"
) +
labs(
title = "Top 5 ZCTAs by Hispanic %"
)
# ------------------------------------------------------------
# TOP 5 BLACK NH
# ------------------------------------------------------------
map_black <- base_map +
geom_text(
data = top_5_black,
aes(
X,
Y,
label = label_text
),
fontface = "bold"
) +
labs(
title = "Top 5 ZCTAs by Black (NH) %"
)
# ------------------------------------------------------------
# DISPLAY FINAL MAPS
# ------------------------------------------------------------
grid.arrange(
map_hispanic,
map_black,
ncol = 2
)
End
Saturday, July 11, 2026
Testing racial predictions with BISG
The common algorithm for estimating race from names and geography is BISG (Bayesian Improved Surname Geocoding). As an example, suppose there is a customer Mary Johnson who lives in Essex County, NJ. If we want to predict her race only from her surname, we use nationwide Census probabilities such as P(Black ∣ Johnson) = 0.3441, P(White ∣ Johnson) = 0.5438, etc.
To improve the prediction, we add her county, using geographic weights such as the relative density of each race in Essex compared with nationwide. The resulting probability P(Black | Johnson & Essex) = (Black national prob × Black geographic weight) / ∑race (race national prob × race geographic weight) = 0.7393.
Finally, we add her first name. Under a Naive Bayes assumption that surname, location, and first name are conditionally independent given race, Bayes’ rule allows us to multiply the components: P(race ∣ surname & geo & first ) ≈ P(race) x P(surname ∣ race) x P(geo ∣ race) x P(first ∣ race). The probability P(Black ∣ Johnson & Mary & Essex) becomes 0.7158.
A summary of these probabilities is:
| Model Type | White | Black | Hispanic | Asian | Other |
|---|---|---|---|---|---|
| Surname Only | .5438 | .3441 | .0272 | .0074 | .0774 |
| Surname+Geo | .1766 | .7393 | .0244 | .0046 | .0551 |
| Surname+First+Geo | .2481 | .7158 | .0042 | .0013 | .0306 |
These calculations can be done with the R package wru (Who are you?). You will need a Census API key to download the Census data, available from https://api.census.gov/data/key_ssignup.html .
I was interested in measuring the accuracy of the BISG predictions. It is not easy to find real (not simulated) data linking name and race. However, several states make their voter registration records public. I requested and downloaded Florida voting registration data from Harvard Dataverse ( https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/UBIG3F ). I limited my study to a single Florida county, Lee, which had over 500,000 registered voters.
The Florida voting data required considerable data cleaning. Among the amusing data issues is that some Lee County voters registered with a specialty post office box zip code, or a zip code with a typo, or an out-of-state zip code. The first two were mapped to county level, and the third category was dropped.
There were 545,187 registered Lee County voters, distributed by race as follows:
| Asian | Black | Hispanic | Other | White | NA |
|---|---|---|---|---|---|
| 6912 | 32407 | 66677 | 9435 | 422333 | 7423 |
I performed the same BISG algorithm as above, against the Florida voter registration data for Lee County. The overall accuracy is 88%, but this is misleading because the Florida data is highly imbalanced toward the White race. A better accuracy measure is one that is calculated separately for each race, and I chose Balanced Accuracy = (Sensitivity + Specificity)/2. Sensitivity measures how well the algorithm finds people of that specific race, and specificity measures how well the algorithm avoids misclassifying people of other races into that specific race.
My Balanced Accuracy results of comparing BISG race predictions versus Florida self-identified race data is as follows:
| White | Black | Hispanic | Asian | Other |
|---|---|---|---|---|
| .8226 | .7179 | .9007 | .7251 | .5041 |
I think these results are pretty good for Hispanic and White, not as good for Black and Asian, and of course pretty poor for Other. Other includes Native American, Native Hawaiian, and others, and are a small overall percentage of the county population.
The citation for the Florida voter data is: Sood, Gaurav, 2017, "Florida Voter Registration Data (2017 and 2022)", https://doi.org/10.7910/DVN/UBIG3F, Harvard Dataverse, V2.
Here is my R code to predict Mary Johnson.
library(wru)
library(dplyr)
# 1. Create a 1-row dataset for Mary Johnson in Essex County, NJ (FIPS "013")
test_voter <- data.frame(
surname = "JOHNSON",
first = "MARY",
state = "NJ",
county = "013", # Essex County FIPS
stringsAsFactors = FALSE
)
# 2. Grab the live Census geographic data for New Jersey
nj_census <- get_census_data(
key = "xxxx",
states = "NJ",
age = FALSE,
sex = FALSE
)
# 3. Generate the 3 prediction variations
pred_baseline <- predict_race(voter.file = test_voter, surname.only = TRUE)
pred_geo <- predict_race(voter.file = test_voter, census.data = nj_census, census.geo = "county")
pred_first_geo <- predict_race(voter.file = test_voter, census.data = nj_census, census.geo = "county", names.to.use = "surname, first")
# 4. Format and clean output to perfectly fit the RStudio layout
combined_predictions <- bind_rows(
pred_baseline %>% mutate(model_type = "Surname Only"),
pred_geo %>% mutate(model_type = "Surname+Geo"),
pred_first_geo %>% mutate(model_type = "Surname+First+Geo")
) %>%
rename(
White = pred.whi,
Black = pred.bla,
Hispan = pred.his,
Asian = pred.asi,
Other = pred.oth
) %>%
mutate(across(c(White, Black, Hispan, Asian, Other), ~ round(., 4))) %>%
select(model_type, surname, first, White, Black, Hispan, Asian, Other)
# 5. Display the output summary
print(combined_predictions, row.names = FALSE)
# End of Mary Johnson prediction
##################################################################################
Here is my R code to measure the accuracy of the BISG predictions against Lee County voters. A considerable portion of the code is for data cleaning of the Florida voter registration data.
library(data.table)
library(dplyr)
library(stringr)
library(caret)
library(wru)
voter_file_path <- "xxxx/LEE_20220621.txt"
Sys.setenv(CENSUS_API_KEY = "xxxx")
raw_lines <- fread(
file = voter_file_path,
sep = NULL, # Read the row as a single string to lock layout shifting
header = FALSE,
col.names = "raw_row"
)
wru_ready_data <- raw_lines %>%
mutate(
# Extract structural components securely out of text blocks
surname = str_match(raw_row, "\\d{9}\\s+(\\w+)")[,2],
first = str_match(raw_row, "\\d{9}\\s+\\w+\\s+(\\w+)")[,2],
zipcode = str_extract(raw_row, "\\b3\\d{4}\\b"),
raw_race = str_match(raw_row, "\\b([MF])\\s+(\\d)\\b")[,3]
) %>%
# Filter out empty records or text extraction errors
filter(!is.na(surname) & !is.na(raw_race)) %>%
# Map Florida's state administrative codes to wru categories
mutate(
true_race = case_when(
raw_race == "5" ~ "white",
raw_race == "3" ~ "black",
raw_race == "4" ~ "hispanic",
raw_race == "2" ~ "asian",
raw_race %in% c("1", "6", "7") ~ "other",
TRUE ~ NA_character_ # Automatically standardizes missing responses to NA
)
)
# Clean data of invalid zip codes and reroute PO Boxes to residential ZCTAs
wru_ready_data <- wru_ready_data %>%
# Exclude out-of-state ZIP codes (Georgia 30xxx/31xxx, Alabama 35xxx/36xxx)
filter(!substr(zipcode, 1, 2) %in% c("30", "31", "35", "36")) %>%
# Map missing/typos and PO boxes directly to matching valid residential ZCTAs
mutate(
zipcode = case_when(
# Defunct / Typos / Missing data map to Central Lee County Baseline
zipcode %in% c("fl-NA", "33929", "33932", "33945", "33970") ~ "33901",
# PO Box explicit routing to residential census counterparts
zipcode == "33902" ~ "33901", # Fort Myers PO Box -> Fort Myers Residential
zipcode == "33906" ~ "33907", # Fort Myers PO Box -> South Fort Myers Residential
zipcode == "33910" ~ "33904", # Cape Coral PO Box -> Cape Coral Residential
zipcode == "33915" ~ "33919", # Fort Myers PO Box -> Cypress Lake Residential
zipcode == "33918" ~ "33903", # N. Fort Myers PO Box -> N. Fort Myers Residential
zipcode == "33994" ~ "33928", # Bonita Springs PO Box -> Estero/Bonita Residential
zipcode == "34106" ~ "34102", # Naples PO Box -> Naples Residential
zipcode == "34133" ~ "34135", # Bonita Springs PO Box -> Bonita Residential
zipcode == "34136" ~ "34135", # Bonita Springs PO Box -> Bonita Residential
TRUE ~ zipcode # Keep all other valid residential ZCTAs as they are
),
# Ensure wru recognizes the geographic county boundary for the fallback imputation
county = "12071" # FIPS code for Lee County, FL
)
nrow(wru_ready_data)
table(wru_ready_data$true_race, useNA = "always")
lee_county_test <- as.data.table(wru_ready_data) %>%
filter(!is.na(true_race)) %>%
mutate(
state = "fl",
surname = as.character(surname),
first = as.character(first),
zcta = as.character(zipcode) # Required column label mapping for ZCTA use
)
florida_zcta_layers <- wru::get_census_data(
key = Sys.getenv("CENSUS_API_KEY"),
state = "FL",
age = FALSE,
sex = FALSE,
census.geo = "zcta",
county.list = NULL # Required parameter boundary choice for independent ZCTAs
)
predicted_lee_county <- wru::predict_race(
voter.file = lee_county_test,
census.surname = TRUE,
surname.only = FALSE,
census.geo = "zcta",
census.data = florida_zcta_layers,
impute.missing = TRUE, # Forces wru to fall back to County priors if needed
skip_bad_geos = TRUE
)
# "53233 (10.1%) individuals' last names were not matched, but ZCTA's baseline geographic racial demographics to calculate the prediction.
evaluation_ready_data <- predicted_lee_county %>%
rowwise() %>%
mutate(
# Isolate which of the five columns held the highest posterior probability string
max_prob_col = c("white", "black", "hispanic", "asian", "other")[
which.max(c(pred.whi, pred.bla, pred.his, pred.asi, pred.oth))
]
) %>%
ungroup() %>%
# Convert fields into factor arrays to prevent tracking sequence errors
mutate(
true_race = factor(true_race, levels = c("white", "black", "hispanic", "asian", "other")),
predicted_map = factor(max_prob_col, levels = c("white", "black", "hispanic", "asian", "other"))
)
# Compute the final metric confusion matrix tracking loop
accuracy_report <- confusionMatrix(
data = evaluation_ready_data$predicted_map,
reference = evaluation_ready_data$true_race
)
print(accuracy_report$overall["Accuracy"]) # Overall model precision
print(accuracy_report$byClass[, "Balanced Accuracy"]) # Success broken down per group
End