Overlapping Array of Population Estimates Create a Fog of Data

By John Besl

In September 2010, the U.S. Census Bureau issued their Vintage 2009 series of population estimates for cities and villages across the country.  In the 17 months since then, the Census Bureau has released several more numbers representing total population, including the “official” 2010 Census count, for these same areas.  For larger municipalities such as Cincinnati, the Census Bureau has now issued eight separate total population figures since September 2010 for various time frames covering the latter part of the 2001-2010 decade.  Sound confusing?  Well, it certainly can be – but in this post I’ll try to explain briefly where all these numbers are coming from, and make comparisons with illustrations featuring local municipalities. 

The most prominent and reliable among all the population numbers produced recently by the Census Bureau for any specific municipality is, of course, that area’s 2010 Census count.  Although it may be flawed, the decennial census produces a population number that comes from an actual enumeration rather than a statistical model.  While it may not be a 100% accurate representation of a city’s population as of the census reference date (April 1, 2010), the census count is the gold standard for population numbers. 

Local officials need information on the number of residents more than once per decade, so the Census Bureau developed the Population Estimates Program (PEP), which produces annual estimates of total population along with detail on demographic characteristics such as age, sex, race, and Hispanic origin for the 50 states, the District of Columbia, and 3,141 counties or county equivalents.  For subcounty jurisdictions like cities, villages, and townships, PEP produces annual estimates of total population only.  Estimates produced by PEP have a reference date of July 1, and are released in a time series called a vintage, with vintage referring to the final year of the time series.  As mentioned above, the Vintage 2009 series was released in 2010 and covers annual estimates of July 1 resident population for the years 2000 through 2009.  This program has been the prime source of demographic information on states, counties, and local areas between the decennial censuses. 

The American Community Survey (ACS) was introduced in the middle of last decade to provide data on population characteristics of local communities every year.  From this ongoing survey the Census Bureau has been producing period estimates of demographic, social, economic, and housing characteristics, with the length of the period determined by population size.  The smallest areas, such as census tracts and municipalities with population size under 20,000, receive five-year period estimates, since it can take 60 months to build up enough sample cases to produce estimates with acceptable reliability.  Note that ACS produces five-year period estimates for all areas, not just the smallest.  Three-year period estimates are available for areas with population between 20,000 and 65,000, as well as larger areas.  Counties and municipalities with population over 65,000 will be deluged with ACS data each year, with the release of five-year, three-year, and one-year period estimates.  In the late months of 2011, then, social science researchers along with elected officials and top administrators of the City of Cincinnati had the opportunity to peruse three new ACS data sets: 2006-2010 five-year estimates, 2008-2010 three-year estimates, and 2010 one-year estimates.  Getting back to my earlier point about the abundance of population figures coming forth from the Census Bureau over the last 17 months, here is a listing of Cincinnati’s population figures from eight distinct Census Bureau data sources, in chronological order by release date:

1. Vintage 2009 Population Estimates – 333,012
2. 2009 ACS 1-year Estimates – 333,013
3. 2007-2009 ACS 3-year Estimates – 333,173
4. 2005-2009 ACS 5-year Estimates – 332,572
5. 2010 Census count – 296,943
6. 2010 ACS 1-year Estimates – 296,907
7. 2008-2010 ACS 3-year Estimates – 298,094
8. 2006-2010 ACS 5-year Estimates – 300,165

Figure 1 below shows the same Cincinnati population number in a slightly different sequence.  The Vintage 2009 (V2009) estimate and 2010 Census form the bookends; ACS estimates in the center are derived from these two sources.  ACS estimates for a period ending in 2009 are derived from the V2009 estimate, while the 2010 Census count forms the foundation for ACS period estimates ending in 2010.  The ACS estimates are ordered somewhat chronologically, by the midpoint of the relevant period.  The midpoint of the 2005-2009 period is 2007, which precedes 2008, the midpoint of the 2007-2009 period, and so on.  Note the huge difference between the two estimates centered on 2008 and the even larger gap in the estimates centered on 2009.  This discrepancy was caused by a substantial over-estimate of Cincinnati’s population in the V2009 estimate series.  This apparent error had its roots back in 2006, when the Census Bureau accepted the City of Cincinnati’s challenge to its original Vintage 2005 estimate of 308,728 as of July 1, 2005.  The city’s successful challenge won a revision of the July 1, 2005 estimate up to 331,310, an instantaneous population gain of 7.3%.  This revision was carried through into subsequent vintages including V2009, but the results of the 2010 Census did not vindicate the decision to revise Cincinnati’s mid-decade population estimate to that degree.

Figure 1. Total Population from Selected Census Bureau Sources: Cincinnati, Ohio

Cities smaller than Cincinnati don’t qualify for quite as many ACS period estimates, due to the population size limitations outlined above.  Florence, Kentucky, with a population in the range of 20,000 to 65,000, receives three-year and five-year period estimates from ACS each year.  Figure 2 below shows the distribution of Florence population estimates that have been issued by the Census Bureau over the past two years.  The progression in ACS estimates looks more reasonable than the Cincinnati chart above, although there is still a sizable discrepancy between the 2007-2009 3-year estimate and the 2006-2010 5-year estimate, both centered on the year 2008.  In an area like Florence that’s gaining population, this result is actually quite reasonable, since ACS period estimates are not supposed to represent the middle year of the period, but the entire span of three or five years.

Figure 2. Total Population from Selected Census Bureau Sources: Florence, Kentucky

We’ve now seen estimates for two areas with differing patterns of population change – decline in Cincinnati and vigorous growth in Florence.  I wondered what a comparable chart would look like for an area experiencing very rapid population growth, and Figure 3 below provides an answer.  Between 2000 and 2010, Independence, KY grew by 65%.  Given the nuances of period estimates, the sequence of ACS estimates for Independence looks very plausible.

Figure 3. Total Population from Selected Census Bureau Sources: Independence, Kentucky

Figure 4 portrays another rapidly growing community in northern Kentucky, the city of Cold Spring.  A much smaller community than Florence or Independence, Cold Spring qualifies only for five-year ACS estimates.  The estimates in Figure 4 are admittedly a little puzzling.  With only a scant difference between the V2009 estimate and 2010 Census count, it’s difficult to understand why the 2006-2010 ACS estimate dropped below 5,500.  The anomalous pattern in Figure 4 is even more difficult to explain when it’s noted that Cold Spring actually witnessed very strong population growth of 55% between 2000 and 2010.  Very small ACS sample sizes probably contribute to the anomaly.  Fewer than 200 housing units in Cold Spring were sampled in either the 2005-2009 or 2006-2010 five-year periods.

Figure 4. Total Population from Selected Census Bureau Sources: Cold Spring, Kentucky

The variation in estimates for a truly small community can be seen in Figure 5.  Felicity, Ohio in Clermont County serves as the illustration.  This small village lost 11% of its population between 2000 and 2010, but the Vintage 2009 estimate erroneously showed marginal growth, from 922 in 2000 to 932 in 2009.  The ACS estimates, on the other hand, suggest a community that’s in a steep population decline.  The extremely small size of Felicity probably accounts in some part for the disproportionate variability of its ACS estimates. 

Figure 5. Total Population from Selected Census Bureau Sources: Felicity, Ohio

In this post, I’ve limited the discussion to the most basic demographic indicator of all – total population.  We’ve seen that ACS cannot always deliver stable estimates.  Demographers have known for years that absolute percent error in population estimates is likely to be higher for small areas or those experiencing fast rates of growth or decline.  

In my next post, I plan to examine age composition data coming out of the ACS, using many of the same municipalities highlighted here to illustrate both reasonable and questionable age distributions from ACS.  Lots of local planning revolves around the age composition of cities and smaller geographic areas, and for many of these places the American Community Survey is the only source for that information.