US Census Data
James Keener
2016-07-19 @ Code and Supply
Established by our Constitution
The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.-Article 1, Section 2 of the United States Constitution
A wealth of Information
Use with Care
Topics in This Talk
- Geographic Hierarchy
- Surveys
- Getting the Data
- Other Data
- Tools
Geography
Geographies
Geographies
FIPS
Not all geographies have FIPS codes, e.g. Legislative Districts
GNIS (from USGS) often cross referenced
Hierarchical, fixed-size segments
Elements at the same level in the hierarchy have different lengths, which can't be obtained by adding parts from lower down in the hierarchy
Geographies
FIPS
420030103003017
- State
- County
- Tract
- Block Group
- Block
FIPS
4219170
- State
- School District
FIPS
UnSD: 4219170
VtD : 420035880
CouSub: 4200361000
Tract: 42003010300
Geography Non-Aligning
Geography Non-Aligning
Geography Non-Aligning
Methods of Correction
- Areal Interpolation
- Projects population values based upon overlapping area
- Fails when population is not evenly distributed by area
- Household/Population Ratio
Interpolation
- Not available at the Block Level
- Census provides percentage of population or households that overlapped
- Fails when measurement is not distributed evenly by area
Sometimes you just have to throw in the towl on some areas
Geography Non-Aligning
Surveys
The Different Surveys
| Decennial Census | American Community Survey |
|---|---|
| Once every 10 years | 1yr- and 5yr- Averages |
| Enumeration | Sampled |
| Basic Demographics | Detailed Demographics |
| Block Level | Often Tract Level |
| "Short Form" | "Long Form" |
Should always compare the Same Survey!
Getting the Data
Getting the Data
Per state can be in the 10s of GBs
Getting the Geographic Data
Importing Raw Data
"Geoheader" is a fixed-width format
Data files are CSV with no header
Data dictionary can be used to build field names and convert short, cryptic ids to English (P016D003 -> Population in Households by Age (Asian Alone Householder) > Total > 18 years and over)
Geographies are Shape Files since 2007
PostgreSQL
Couple repos to create schemas, download data, and import the data
With proper indices, even complex queries are very fast
Other Datasets
Other Datasets
- Economic Census
- Longitudinal Employer-Household Dynamics
- American Housing Survey
- American Time Use Survey
- Rental Housing Finance Survey