Expert Witness Profiler

Tag: Statistical analysis

  • Psychology Expert Witness’ Testimony About the Research on the Reliability of the Field of Firearm and Toolmark Identification Admitted

    Psychology Expert Witness’ Testimony About the Research on the Reliability of the Field of Firearm and Toolmark Identification Admitted

    Defendant Adrian Garcia was charged in a three-count indictment for an alleged carjacking that occurred on May 22, 2022:

    1. Carjacking
    2. Using and carrying a firearm during and in relation to a crime of violence, and possessing a firearm in furtherance of such crime, and discharging said firearm
    3. Being a felon in possession of a firearm and ammunition.

    The Government stated it would present evidence showing the offender possessed a firearm, used it during the carjacking, and discarded it while fleeing. Police later recovered the firearm at the scene.

    The Government retained firearms and toolmark expert witness Erich Smith. Smith examined a spent cartridge case found at the carjacking scene and a nearby 9mm pistol. He determined that the cartridge case was fired from the recovered pistol. Smith followed the E3CV methodology, which stands for evaluation, classification, comparison, conclusion, and verification. This method is recognized by the Association of Firearm and Toolmark Examiners (AFTE). After a challenge under Rule 702 and Daubert, this Court permitted Smith to testify.

    To rebut Smith’s testimony, Defendant retained psychology expert witness, Nicholas Scurich, Ph.D., who specializes the evaluation of forensic firearm/toolmark examination. Now, the defense intended to use Scurich’s testimony to impeach or rebut Erich Smith’s testimony.

    To begin with, Scurich was supposed to testify regarding low repeatability and reproducibility of conclusions by firearm/toolmark examiners in validation studies. Moreover, to the extent Smith relies on validation studies in his testimony, Scurich may testify to fundamental design flaws in the studies regarding the sampling of examiners and materials, problem difficulty and bias, missing data, and inconclusive responses. Finally, Scurich will rebut Smith’s testimony by testifying about reasons for growing skepticism by scientists in the field.

    Psychology expert witness

    Psychology Expert Witness

    Nicholas Scurich is a quantitative psychologist with a PhD in Psychology from the University of Southern California. His training included courses on advanced research methods and advanced statistical methodology. He currently works as a professor at the University of California at Irvine, where he is the chairman of the psychology department. Scurich teaches graduate courses in advanced research methods and forensic assessment. Moreover, he has authored more than 75 peer-reviewed journal articles, book chapters, law review articles, technical reports, and conference proceedings.

    Want to know more about the challenges Nicholas Scurich has faced? Get the full details with our Challenge Study report.  

    Discussion by the Court

    The Government moves to exclude Scurich’s testimony for multiple reasons. The Government argued that the Notice is too broad and did not comply with Federal Rule of Criminal Procedure 16. It also contends that Scurich’s testimony should be excluded under Federal Rule of Evidence 702 because Defendant has not established that Scurich is qualified or that his opinions are reliable, relevant, or helpful to the jury. According to the United States, allowing his testimony would undermine the Court’s gatekeeping function under Daubert and confuse the jury. For the reasons explained herein, the Court disagrees with the Government.

    A. Rule 16 does not justify the exclusion of Scurich’s testimony

    The United States argues that Defendant’s Notice does not contain “a complete statement of all opinions” that Scurich will offer, but instead too broadly lists topics, rather than substantive opinions. It asserts his proposed testimony should be struck on this ground alone. 

    While Defendant’s Rule 16 summaries are relatively cursory, the record contains substantial discussion of Dr. Scurich’s opinions based on his published articles.

    Also, Defendant provided a list of the 25 studies that Scurich is familiar with that informs the Government about which validation studies he may testify. Moreover, any prejudice from the lack of more specificity in the Notice was cured at the hearing. Basically, the purposes of Rule 16 include minimizing surprise from unexpected expert testimony and allowing a party to prepare for cross-examination and presentation of opposing experts.

    Therefore, the Court held that Rule 16’s purposes are met via the disclosures in the record and the opportunity the United States had to cross-examine Scurich at the Daubert hearing. Basically, the United States asked Scurich specific questions as to each of the five enumerated opinions listed in the Notice and gained additional detail as to each. 

    B. Defendant satisfied his burden under Rule 702 and Daubert to establish Scurich’s qualifications and the relevance and reliability of his opinions

    Now, the Federal Rule of Evidence 702 governs the admissibility of expert testimony. An expert must be qualified by knowledge, skill, experience, training, or education, and their testimony must be based on sufficient facts, reliable principles, and a reliable application of those principles to the facts of the case.

    The Court finds that Scurich is qualified as an expert in research design, the evaluation of research design, data analysis including statistical analysis, and the evaluation of data analysis.

    1. Scurich is qualified as an expert.

    According to the Government, Scurich is not qualified in the field of firearm and toolmark identification evidence to render an opinion about the evidence in this case. Scurich, however, is not offering an opinion about the firearm and toolmark analysis Smith conducted in this case. Instead, the Court held that his testimony will be more generally about the research on the reliability of the field of firearm and toolmark identification.

    Although he does not have experience and training in firearms and toolmark casework, he has extensive experience and training in research methods and design. 

    2. Scurich’s opinions are reliable.

    Moreover, the Court held that he has extensively reviewed the literature and research studies testing the firearms and toolmark field and analyzed the raw data therein. He applied reasoning to reach his conclusions based on his specialized knowledge in research design and statistical analysis, his education, and his training.

    3. Scurich’s opinions are relevant and helpful to the jury.

    The United States next argued that, because the issues at trial are not about research designs or the best way to analyze statistical data, the evidence is not relevant and will confuse the jury. 

    The Court held that Scurich’s testimony will aid the jury in understanding the issue of the reliability of firearms and toolmark forensic analysis and help the jury evaluate the weight to afford the expert testimony of Smith.

    4. The Court will permit Scurich to offer expert testimony.

    Defendant has demonstrated that Scurich is qualified, his opinions are reliable, and his testimony is relevant. Therefore, the Court will allow Scurich to testify as an expert witness at trial.

    Held

    In conclusion, the Court denied the United States’ motion to exclude the testimony of psychology expert witness Nicholas Scurich.

    Key Takeaway:

    Scurich has sufficient education, training, experience, and knowledge to qualify as an expert in research design, the evaluation of research design, data analysis including statistical analysis, and the evaluation of data analysis to offer the opinions.  Also, he has acquired familiarity with the research studies and underlying data that the firearm and toolmark examiners rely on to support their assertion that the AFTE theory of identification is a reliable method for comparing cartridge cases and bullets. 

    Case Details:

    Case Caption: United States V. Garcia
    Docket Number: 1:22cr1171
    Court: United States District Court for the District of New Mexico
    Dated: December 10, 2024
  • Court validates the objective findings of the statistics expert witness in this class action lawsuit consisting of disability discrimination claims

    Court validates the objective findings of the statistics expert witness in this class action lawsuit consisting of disability discrimination claims

    This case originated from a class action lawsuit filed by the Center for Independence of the Disabled, New York and other nonprofit organizations (collectively referred to as Plaintiffs) against the Metropolitan Transportation Authority (“MTA), New York City Transit Authority (“NYC Transit”), and the City of New York(“the City”) (collectively referred to as Defendants). The lawsuit alleged that hundreds of thousands of New Yorkers with mobility disabilities were persistently and discriminatorily excluded from accessing the New York City subway system due to the failure of the Defendants to adequately maintain the already limited number of elevators in the system. 

    The Plaintiffs contended that subway riders with mobility disabilities routinely faced abrupt and extended elevator outages lasting for months, without any prior notice or warning mechanisms like signage or announcements. Moreover, the Defendants allegedly failed to provide any alternative accommodations to facilitate the transportation of people requiring elevator access when outages occurred. The Plaintiffs asserted that the overall elevator accessibility in the New York City subway system was already poor compared to other major US cities, with only 112 (24%) of the 472 subway service line stations in New York City being wheelchair-accessible.

    Public data from the MTA demonstrates that over 9,019 elevator outages occurred during the one-year period ending on June 30, 2015. Over 4,100 of those outages were unscheduled—a predictable result of Defendants’ failure to maintain and implement adequate preventative maintenance procedures

    Plaintiffs alleged that Defendants’ failure to maintain the limited number of elevators they provide in the New York City subway system violated federal and local disability rights laws, including Title II of the Americans with Disabilities Act (“ADA”), Section 504 of the Rehabilitation Act of 1973, and the New York City Human Rights Law (“NYCHRL”). The Court was left to assess whether Defendants have established that they provide reasonable accommodations to class members when elevator outages prevent them from accessing the subway.

    The Defendants retained Dr. Alan J. Salzberg, a statistician, to analyze the overall median availability of elevators in the NYC subway system using MTA data. The Plaintiffs did not contest his availability calculations but argued that the aggregated figures could obscure variability. Hence, the Plaintiffs presented two rebuttal experts – Andrew D. Schwarz and Sylvia Morse – to provide context to Dr. Salzberg’s opinions. 

    The Plaintiffs also filed a motion to exclude Dr. Salzberg’s testimony, arguing that he exceeded his expertise scope, manipulated data unreliably, and made flawed assumptions, rendering his testimony unhelpful and misleading. 

    Statistics Expert Witness

    Alan J. Salzberg Ph.D., is Senior Statistician and Principal of Salt Hill. His focus is statistical analysis, sampling, estimation, and modeling, especially using large or complex datasets. Many of Dr. Salzberg’s consulting projects and research papers have related to the detection and measurement of bias. He has testified as an expert witness in statistics in federal and state court. Prior to joining Salt Hill, Alan was CEO of Analysis & Inference. Salzberg holds a Ph.D. in Statistics from the Wharton School of the University of Pennsylvania, where he also received a Bachelor of Science in Economics.

    Discussions by the Court

    In response to Plaintiffs’ motion to exclude the testimony of Defendants’ expert, Alan Salzberg, the Court engaged in a detailed discussion rooted in the parameters set by Federal Rule of Evidence 702. The Court emphasized that, under Rule 702, the party seeking to introduce expert testimony bore the burden of demonstrating that the testimony met certain criteria, including being based on sufficient facts or data, the product of reliable principles and methods, and reliably applied to the facts of the case. Additionally, the expert’s testimony had to be relevant and assist the jury in understanding the case. 

    The Court considered the motion to exclude the testimony of Alan Salzberg, the Defendants’ expert, who had submitted reports critiquing two of the Plaintiffs’ experts, Andrew Schwarz, an economist, who developed a model to demonstrate the practical implications of elevator outages on passengers traveling on certain high-usage subway routes during rush hours, and Sylvia Morse, Policy Program Manager at Pratt Center for Community Development. The Court began by addressing Salzberg’s expert report titled “Subway Accessibility Analysis.” Salzberg, was tasked with reviewing public elevators in the New York City subway system to determine the elevator, station, and station stop availability for riders with mobility disabilities. Schwarz’s analysis was “intended to highlight the impact of the frequency of inaccessibility calculated by Salzberg on the commutes of targeted types of individuals.”

    In response to Schwarz’s report, Salzberg submitted a reply report on May 2019, using the same Metropolitan Transportation Authority’s (“MTA”) Trip Planner tool that Schwarz relied on for his analysis. Salzberg performed his own analysis of the 200 trips considered by Schwarz and identified errors in Schwarz’s analysis, such as, Schwarz had not selected the 200 subway trips at random and excluded downtown Manhattan stations.

    After the Second Circuit’s remand, Salzberg submitted an update to his report, dated September 12, 2022, concerning accessible elevator, station, and station stop availability, using new data, from 2019 through 2021. In response, Plaintiffs submitted an updated report from Schwarz, dated November 23, 2022. Schwarz, using Salzberg’s statistical calculations, again examined the likelihood that a commuter taking a trip along 200 commuting routes involving subway stations commonly used by passengers with mobility disabilities would experience a failed trip during a particular period of time. The Court noted that Plaintiffs did not challenge Salzberg’s availability calculations, focusing instead on his qualifications and the assumptions underlying his statistical calculations. 

    Plaintiffs argued that Salzberg was unqualified to render an expert opinion in the case due to his lack of expertise in transportation systems, commuting patterns in New York City, or accessibility barriers for people with mobility disabilities. The Court countered this by highlighting that Salzberg analyzed the same data examined by Schwarz, who himself was an economist without expertise in transportation systems. The Court emphasized that Salzberg’s qualifications were appropriate for critiquing Schwarz’s model.

    Another argument raised by Plaintiffs was the unreliability of Salzberg’s methodologies, grounded in alleged flawed assumptions. Plaintiffs contended that these flaws rendered his opinion unreliable and unhelpful to the trier of fact. The Court, however, clarified that criticisms of methodology went to the weight of the expert’s opinion, not its admissibility. The Court stated that faults in methodology were a matter for cross-examination. 

    Plaintiffs specifically challenged Salzberg’s assumption that the MTA’s Trip Planner accurately reflected availability and delays associated with having to reroute or that passengers received timely notification of elevator outages. The Court deemed it reasonable for Salzberg to have relied on data pertaining to route availability and delays, which were maintained by the MTA in the regular course of business and were accessible to the public through the MTA’s Trip Planner online tool. The Court also noted Schwarz’s reliance on the same Trip Planner data in his analysis, highlighting that any inaccuracies would affect both analyses. Plaintiffs’ claims about data inaccuracy were considered arguments concerning the weight of the evidence and not to its admissibility.

    Another point of contention was Salzberg’s definition of a “successful” trip, where Plaintiffs argued that he deemed a trip successful regardless of its duration or onerousness. Salzberg’s use of the Trip Planner tool to determine alternative itineraries was defended by the Court, emphasizing that Plaintiffs could challenge the feasibility of these alternatives on cross-examination. 

    The Court also addressed Salzberg’s consideration of nearby stations when an elevator outage affected the original station, adding travel time to the itinerary whenever necessary. Plaintiffs contended that this introduced selection bias. The Court rejected this argument, stating that Salzberg reliance on the MTA’s Trip Planner to determine alternative itineraries demonstrated no selection bias.

    Moving to the critique of Plaintiffs’ expert Sylvia Morse, the Court outlined the situation. Morse had submitted a rebuttal report focusing on the real-world performance of the bus system and
    factors affecting rider experience, challenging two assumptions made by Salzberg. The first assumption was that transit users with mobility disabilities could always access buses, and the second was that the MTA’s Trip Planner accurately reflected a transit user’s travel time. Ultimately, Morse concluded that accessibility barriers and the unreliability of buses impeded or delayed the completion of alternative itineraries that relied on use of the bus system for transit users with mobility disabilities.  

    In response, Salzberg prepared a rebuttal report criticizing Morse’s reliance on anecdotal customer complaints and arguing that such complaints represented a tiny fraction of all accessible bus trips and was not a statistically valid basis to support a conclusion that the bus system suffered from widespread barriers to accessibility. Plaintiffs challenged Salzberg’s analysis, raising arguments that the Court deemed relevant to the weight of the evidence, not its admissibility.

    Plaintiffs faulted Salzberg for assuming that each complaint concerned only a single ride or bus operator. The Court noted that Plaintiffs had no evidence to support their assumption that a complaint could pertain to multiple trips, highlighting the lack of basis for this criticism. Plaintiffs’ challenge to Salzberg’s conclusion regarding Trip Planner’s notification of elevator outages in real time was also addressed. The Court acknowledged Plaintiffs’ freedom to challenge the persuasiveness of this opinion on cross-examination but found the assumption not unfounded enough to render the opinion inadmissible.

    Held 

    The Court denied the Plaintiffs’ Daubert motion to exclude Dr. Salzberg’s testimony, finding that their criticisms went to the weight rather than the admissibility of his opinions. The Court determined there was no basis to preclude Dr. Salzberg’s expert testimony under the applicable rules of evidence. The Court has not arrived on an outcome for this case since the remaining issues involved in this case still await resolution.

    Key Takeaways

    This case highlights several important considerations regarding the admissibility of expert witness testimony. A key takeaway is that critiques of an expert’s methodology ,sometimes, generally relate to the weight of the testimony rather than its admissibility. The Court emphasized that alleged flaws in an expert’s assumptions or analysis are fodder for cross-examination but do not necessitate exclusion. Furthermore, the ruling demonstrates that rebuttal experts need not have specialized expertise in the substantive field at issue if they are commenting on another expert’s methodology. Finally, the Court highlighted that rebuttal experts should bring specialized knowledge to bear on an issue rather than just speculating or relying solely on anecdotal evidence. These insights shed light on key considerations for expert witness testimony.