Expert Witness Profiler

Category: Data Analysis Expert Witness

  • Court Excludes Data Analysis Expert Witness’ Opinions on the Issue of Fair Housing

    Court Excludes Data Analysis Expert Witness’ Opinions on the Issue of Fair Housing

    Plaintiff, the Fair Housing Center of Central Indiana (“FHCCI”) sued Defendants, M&J Management Company, LLC, d/b/a The Sexton Companies, Sexton Carlyle, LLC, Remington Court, LLC, and Sexton Windsor, LLC (collectively, the “Defendants” or “M&J Management”) based solely on Defendants’ occupancy policy requiring no more than two occupants per bedroom (“Defendants’ Occupancy Standard”).

    FHCCI claimed Defendants’ Occupancy Standard discriminates based on familial status in violation of the federal Fair Housing Act (“FHA”), and the Indiana Fair Housing Act (“IFHA”), by creating a disparate impact.

    The Fair Housing Center of Central Indiana is a private, non-profit Indiana corporation headquartered in Indianapolis that provides fair housing services throughout the state. Plaintiff’s mission is to ensure and enhance equal housing opportunities by eliminating housing discrimination through advocacy, enforcement, education, and outreach. Plaintiff receives 1,650 to 1,800 complaints and inquiries per year from the public, though typically only 30-40 complaints per year relate to minor children.

    Plaintiff hired James Colbert to tabulate census data and analyze whether Defendants’ occupancy standard has a disproportionate effect on households with minor children. Colbert is a data manager and analyst for the Polis Center. 

    Defendants argued that Colbert lacked proper training and qualifications to render an opinion regarding causation in a fair housing case, his report consisted of unsupported factual and legal conclusions, and his testimony was unreliable and unhelpful.

    When the Defendants initially filed a motion to preclude Colbert, the Court recognized their concerns, agreed that the value of Colbert’s testimony was uncertain, but nevertheless denied the motion. Subsequently, Defendants renewed their motion to strike Colbert.

    Data Analysis Expert Witness

    James Colbert has been employed by the Polis Center since February of 2004 in one capacity or another, including as a student intern, full time
    employee and contractor.

    During the last 19 years he has worked with many data sets including Census data. His experience with Census data includes collection of published data, analysis, transformation, and dissemination.

    Get the full story on challenges to James Colbert’s expert opinions and testimony with an in-depth Challenge Study. 

    Discussion by the Court

    James Colbert

    To perform his analysis, Colbert used the American Community Survey (ACS) Public Use Microdata Samples (PUMS). The ACS data is collected from a data sample of about 1% of the U.S. population through a yearly survey collected by the U.S. Census using questionnaires. PUMS data is public microdata that looks at individual-level responses to the ACS.

    When reviewing PUMS data, an analyzer can only review data from the state level or from a Public Use Microdata Area (PUMA), which is an area that contains at least 100,000 people. PUMS data only accounts for a limited number of variables for data collected from households: tenure (owned or rented), number of people in households, age of occupants, rents, and rent adjustment variables. There are no variables for square footage of the household, type of household (apartment or house), amenities, public transportation, or a tenant’s preference. The surveys rely on self-reported data, so the data is only accurate if the respondent answered accurately. 

    He examined data from 2017 to 2021. Colbert considered whether rental households with minor children would be disproportionally affected by a landlord’s occupancy standard of two persons per bedroom by calculating a “risk for disparity ratio.” He concluded that for the three apartment complexes in question, the percentage of children living at the apartments are lower than the percentage of children living in rental households in larger geographies in which the complexes are contained. 

    Analysis

    To establish a prima facie case of disparate impact, Plaintiff must first identify an outwardly neutral policy and compare how that policy affects a protected class compared with others.

    Defendants’ occupancy standard is an outwardly neutral policy that impacts a protected class, because it excludes households with three or more minor children from renting at Defendants’ apartment complexes. However, Plaintiff’s prima facie case must go one step further and demonstrate the essential element of robust causality. 

    Plaintiff claimed that because households with more occupants tend to include families with minor children, and Defendants do not have a high percentage of families with children residing at the apartments in question, then Defendants’ occupancy standard must cause the low percentage of families with children, and there can be no other explanation.

    The Court held that Plaintiff failed to demonstrate robust causality. Plaintiff largely relied on testimony from its expert, James Colbert. 

    Qualification

    The Court held that Colbert is not qualified to give opinion on the issue of fair housing. He is not a statistician or an expert in statistics, and he has no specific training or work experience on housing, occupancy standards, fair housing, housing discrimination, housing laws, or managing and owning multifamily housing. Colbert and Plaintiff failed to explain how Colbert’s experience in data management leads to his conclusion that Defendants’ occupancy standard had a disparate impact on families with children, why his experience is a sufficient basis for the opinion, or how Colbert’s experience reliably applied to the facts. Colbert had no education related to PUMS data, no experience collecting ACS or PUMA data, and no involvement in the compilation of the data he used to form his opinion.

    Even if the Court found Colbert qualified to give an opinion, his opinions were excluded because his methodology is not based on sufficient facts and data. Colbert admitted his techniques were never subject to peer review or publication. Neither Plaintiff nor Colbert presented evidence related to the known potential rate of error to Colbert’s techniques.

    Methodology

    When making his findings, Colbert only considered four variables: whether someone rented or owned, the age of occupants, rent, and rent adjustment variables. Colbert never looked at floor plans of Defendants’ properties, and he knew nothing about their square footage. The Court found that Colbert failed to consider any extraneous variables or potential key relevant factors that might have explained the lack of families with children choosing to live at Defendants’ properties, such as less square footage, small appliances, school districts, or parking.

    Colbert did not compile the PUMS or ACS data he relies on, so he could not account for its veracity. In addition, when comparing this data with Defendants’ properties, Colbert relied on a snapshot in time: tenant rosters for the properties at issue on August 9, 2022. Tenants renting apartments change daily, and without any further data points, it is impossible to know if the data he relied on was consistent with any point in time beyond that moment on August 9.

    The Court held that the analytical gaps between the data and Colbert’s conclusions are too great. Colbert’s methodology is not limited to prospective tenants or actual applicants. Rather, his analysis assumes all families with children who currently rent (in any type of housing) want to live at Defendants’ apartments but were excluded from doing so, which resulted in a disparate impact on many families with children. However, Colbert’s methodology resulted in a skewed result, as it is not clear what number of families captured actually desired to live at Defendants’ apartments. Colbert failed to consider any alternative causal variables.

    The Court held that the Plaintiff must present reliable evidence showing that the Defendants’ occupancy policy caused a disparity, but they failed to do so. Colbert explicitly stated that he was not opining on causation.

    Held

    The Court granted Defendants’ renewed motion to exclude Plaintiff’s expert James Colbert. Even if his testimony were allowed, it would not be sufficient to establish a prima facie case of disparate impact under the FHA or IFHA.

    Key Takeaway:

    It was noted that Colbert’s theory and techniques were never subjected to peer review or publication. Colbert had no knowledge as to whether occupancy increasing results in increased vermin, increased wear and tear, or faster devaluation of property. In conclusion, Colbert did not have an opinion on whether Defendants’ policies caused a discriminatory effect. 

    Please refer to the blog previously published about this case:

    Data Analysis Expert Witness’ Testimony Admitted Despite his Failure to Incorporate Extraneous Factors in his Analysis

    Case Details:

    Case Caption: Fair Housing Center Of Central Indiana, Inc. V. M&J Management Company, Llc, D/B/A The Sexton Companies Et Al
    Docket Number: 1:22cv612
    Court: United States District Court, Indiana Southern
    Order Date: February 5, 2025
  • Data Analysis Expert Witness’ Testimony Admitted Despite his Failure to Incorporate Extraneous Factors in his Analysis

    Data Analysis Expert Witness’ Testimony Admitted Despite his Failure to Incorporate Extraneous Factors in his Analysis

    Defendants, M&J Management Company, LLC, d/b/a The Sexton Companies own and operate multiple multi-family housing units in Indianapolis and St. Joseph County, Indiana. Defendants have an occupancy standard of two occupants per bedroom that applies to all floor plans, regardless of the age of the occupants. Plaintiff, Fair Housing Center of Indiana alleged that Defendants’ occupancy standard violates the Fair Housing Act and Indiana Fair Housing Act because it discriminates based on familial status. 

    Plaintiff proffered James Colbert as its expert to analyze whether Defendants’ occupancy standards have a disproportionate effect on households with minor children. While Defendants argued that Colbert’s findings were based on flawed logic and incomplete analysis, they waited until July 30, 2024, to move to preclude Colbert’s Rule 26 expert report and to exclude his testimony in this case.

    Data Analysis Expert Witness

    James Colbert has a master’s degree and has worked within the professional field of data analysis, including in-depth analysis of census data, for over 20 years.

    He works as a data manager at The Polis Center at Indiana University Purdue University Indianapolis.

    Get the full story on challenges to James Colbert’s expert opinions and testimony with an in-depth Challenge Study. 

    Discussion by the Court

    Defendants challenged Colbert’s expert report and testimony, arguing his opinion consisted primarily of unsupported factual and legal conclusions.

    While Defendants filed their motion to preclude Colbert’s testimony too late to address his qualifications at summary judgment, the Court will address Defendants’ concerns now, before trial. Given that the summary judgment standard is no longer at play, the Court is not bound by its summary judgment findings and instead takes a fresh look at Colbert’s report.

    Colbert is not qualified to give expert opinions on the issue of fair housing

    The Court held that Colbert possessed sufficient academic and practical expertise to qualify as an expert in the fields in which he seeks to testify—data analysis and statistical disparity.

    Defendants argued that Colbert is not qualified to give expert opinions on the issue of fair housing. Defendants claimed that Colbert has no experience, training, or education regarding the ownership and management of apartment complexes, the psychology behind tenants choosing where to live, and fair housing laws, and his background as a data analyst cannot support his opinions regarding apartments’ occupancy rules.

    However, the Court observed that Colbert does not offer opinions on topics or rely on any data that would require more extensive knowledge of apartment management, housing choice, or fair housing laws. Rather, Plaintiff acknowledged that it retained Colbert for the sole purpose of “crunching numbers and calculating relative impacts between various demographic groups.” Both sides agreed that Colbert’s background is in data analysis. Colbert’s CV indicates that he has a master’s degree and has worked within the professional field of data analysis, including in-depth analysis of census data, for over 20 years.

    Colbert’s opinion will not assist the trier of fact

    Defendants claimed that Colbert’s report is not helpful and is simply a layman’s opinion, speculative, and based on little more than broad census data.

    Relatedly, Defendants also argued that Colbert will mislead and confuse the jury in this case. However, this matter is set for a bench trial, not jury trial. “In a bench trial, once the Court has fulfilled its gatekeeping function, it becomes a trier of fact that needs to assess the evidence itself—not just the methodology underlying that evidence.” The Court held that whether Colbert will assist the Court, as a trier of fact, is not a concern at this stage.

    Colbert failed to incorporate extraneous factors in his analysis, and he did not consider alternative reasons why a household may decide not to live in Defendants’ apartment complexes

    A closer question, however, is whether Colbert’s methodology is scientifically reliable. Defendants argued that Colbert failed to conduct the testing necessary to determine whether Defendants’ occupancy standard caused a disproportionate exclusion of families with minor children from accessing housing.

    In addition, Colbert failed to incorporate extraneous factors in his analysis, and he did not consider alternative reasons why a household may decide not to live in Defendants’ apartment complexes. Instead, Colbert relied heavily on the opinion that all households in the relevant geographical groups that desired to live at Defendants’ apartment complexes were unable to do so because of Defendants’ occupancy standard.

    Plaintiff argued that the existence of other potential causes or explanations for the low number of households with children at Defendants’ complexes is irrelevant to Colbert’s analysis, because his analysis focused on the predictable effect of the occupancy cap on the rental household population of Indiana, Marion County, and St. Joseph County.

    It was noted that Colbert’s report does not consider potentially key additional relevant factors that might have explained the lack of families with children choosing to live at Defendants’ complexes, such as the fact that these apartments had less square footage, small appliances, no backyard, and limiting parking.

    Defendants’ concerns regarding Colbert’s report are well founded. The fact that Colbert never addresses any potential extraneous factors certainly casts doubt on the reliability of his opinion. However, ultimately, the Court agrees with Plaintiff that these concerns do not completely exclude Colbert from testifying at trial.

    At trial, Defendants—and the Court—will press Colbert on why he failed to consider any alternative explanation or major variable and can test the reliability of his testimony.

    Colbert’s report also does not define “rental household”

    Colbert’s report stated that the percentage of children living at Defendants’ complexes is lower than other rental households in the counties in which the complexes are located, but he does not explicitly define “rental household.”

    It appeared that rental household included all rental dwellings in the respective counties, including rental houses, rental townhomes, rental duplexes, rental lofts, and other rental multi-family apartment complexes with larger square footage, elevated appliances, and more parking options. Nevertheless, the Court held that Colbert’s report did not take into consideration the idea that households with more occupants, which tend to include households with children, may prefer to live in homes with larger square footage than the options available at Defendants’ complexes.

    Court cannot conclusively say Colbert’s calculations are speculative simply because they are grounded in PUMS data and Defendants’ own data about their residents

    First, Defendants did not dispute the existence of their two-per-bedroom occupancy policy. Second, Colbert primarily relied on American Community Survey Public Use Microdata Sample (PUMS) data obtained from the United States Census. Defendants have not offered any argument that using census or PUMS data was improper. Likewise, the Court cannot conclusively say Colbert’s calculations are speculative simply because they are grounded in PUMS data and Defendants’ own data about their residents. Defendants’ concerns, while valid, can be addressed through cross examination at trial, and further inquiry, if needed, from the Court.

    Also, Plaintiff seems to believe it can introduce Colbert’s report as evidence at the upcoming bench trial. While the Court has concluded that Colbert may testify at trial, Plaintiff will cannot introduce the report itself at trial. It is inadmissible hearsay. 

    Colbert can testify about the opinions in his report, and Defendants may use Colbert’s report to cross examine him and attempt to impeach him. However, Plaintiff cannot introduce Colbert’s report as evidence.

    Held

    The Court recognized Defendants’ concerns and agreed that the value of James Colbert’s testimony is uncertain. Nevertheless, the Court denied the Defendants’ motion to preclude Colbert’s testimony. This case remains set for a final pretrial conference at 1:30 pm on October 29, 2024, and for a bench trial on November 12, 2024.

    Key Takeaway:

    Colbert is adept at locating census data and applying statistical methods. However, Defendants’ concerns regarding Colbert’s report are well founded. Colbert’s report does not consider potentially key additional relevant factors that might have explained the lack of families with children choosing to live at Defendants’ complexes, such as the fact that these apartments had less square footage, small appliances, no backyard, and limiting parking.

    Ultimately, the Court agreed with Plaintiff that these concerns do not completely exclude Colbert from testifying at trial. At trial, the Court will be able to weed through the parties’ arguments regarding whether Colbert’s testimony and report demonstrate causation, a predictable effect, or something else.

    Similarly, Colbert’s report also does not define “rental household.” Colbert’s report states that the percentage of children living at Defendants’ complexes is lower than other rental households in the counties in which the complexes are located, but he does not explicitly define “rental household.” 

    Case Details:

    Case Caption: Fair Housing Center Of Central Indiana, Inc. V. M&J Management Company, Llc, d/b/a The Sexton Companies Et Al
    Docket Number: 1:22cv612
    Court Name: United States District Court, Indiana Southern
    Order Date: October 10, 2024