CA Dept. of Education

Home

Culturally Responsive Assessment Archive 2013

jh  

James Hiramoto, Ph.D.,
School Psychologist

James graduated with an MA and PhD in Educational Psychology from UC Berkeley’s, School Psychology Program. He has 17 years of experience as a school psychologist. He advises and provides trainings for superintendents, school administrators, teachers and special education staff. He has over 8 years of experience as a university professor and director, training school psychologist at the Master and Doctoral level. His areas of expertise align with the subjects he teaches and or presents at state or international conferences. These areas include: Cognitive ability, neuropsychological, alternative and culturally responsive assessment; crisis planning, management and intervention; educational research methodology and statistics; program evaluation; consultation and special education law.

Submit A Question

Click a topic below to expand the full question and answer.

  • Cultural and Other Considerations

Question:

James

I am in the process of assessing an African teenager who has lived in the states for about 6 years. The student spoke xxx until the age of 6 and then went into foster care (need to verify this), was taught in only English and was adopted by Americans; CELF-5 scores are xxx.

Could you give me any ideas about these scores and what direction I should go? I still need to talk more with the parents, assess just comprehension, but what about the xxx language and culture that might explain these scores or not.

Just wondering about many things,

Thank you,
Wendy


Answer:

Hi Wendy,

While Larry P. may not have foreseen a case like this, you may wish to consult CSHA’s Practice Guidelines for the Assessment of the African American Child and The Assessment of African American Children: An Update on Larry P. found at this website http://old.csha.org/diversitypapers.cfm. The DCN also has a training for SLPs on this topic. More information can be found under the Professional Development tab at www.dcn-cde.ca.gov. Look under current year trainings for opportunities to attend. This may give you some ideas of what you may wish to use as part of your assessment.

Now, before ascribing cultural factors as a rational for this student’s behavior or performance on standardized tests, consultation with adoptive parents and school psychologist regarding the student’s history would be a good place to start. The circumstances surrounding being taken away from parents and home at age 6; placed into foster care and required to learn a new language (English), and then moved half way around the world to be raised by people you do not know is quite an ordeal for anyone to process. Understanding these environmental factors that came into play during early language development will be important. Having that information will help the IEP team understand how cultural factors influenced the student, given these extenuating circumstances.

I am not a cultural expert for where this student has come from, however finding, a “cultural broker” would be beneficial in this case. A “cultural broker” is an individual who acts as a bridge/mediator between groups or persons of different cultural backgrounds. As a cultural broker, one understands the culture (e.g., values and beliefs) s/he represents and is able to facilitate the bridge of understanding with that of the dominant culture. The adoption agency the parents used might be a good source as they may be able to provide you with a “cultural broker” or point you in the right direction. I have also found a country’s consulate in the US being a good resource. Depending on the country, they may have a local branch office near you or can point you in the right direction.
I hope this has helped.

Sincerely,
James


  • The Assessment of African American Students for Special Education

Question:

James

Is the NEPSY-II a valid option to assess African American students in California? I know it uses info from cognitive measures as evidence of validity which makes me doubt. However, it doesn’t give me a measure of IQ or general ability but valuable information about specific processes such as auditory, visual, and executive functioning, among others.

Jena


Answer:

Hi Jenna,

The NEPSY-II looks at specific processing areas:

  • Attention and Executive Functioning
  • Language
  • Memory and Learning
  • Sensorimotor
  • Social Perception
  • Visuospatial Processing

Looking at specific processing areas is an alternative method approved in the California Department of Education (CDE) Legal Memorandum 1997. Neuropsychological assessment was also included as an alternative assessment method approved by the Larry P. Task Force.

The NEPSY-II makes it very clear in the user’s manual that the entire battery should not be given. Prior to administration, one of the six processing areas above needs to be identified as an area of concern. It is not meant to be an exploratory tool. No single test should be used to determine eligibility. This doesn’t mean that more tests should be used. Rather, tests should be used to support evidence for eligibility. The impact of the processing deficit should be evident in life areas other than the test, as well. If the link between processing area and impacted academics (including life areas where those skills are required) isn’t evident, eligibility for a Specific Learning Disability (SLD) isn’t met.

This NEPSY II is not designed nor can function as a measure of general cognitive ability, or of any other synonym for intelligence. It is not based on a factor structure that would lead to an overall measure. It is a tool to determine if a processing area deficit exists either organically or due to trauma to the brain.

After exploring the norming data and its appropriate use, determining if the NEPSY-II is a valid measure is a question that needs to be answered by your district.


  • NEPSY II as a Replacement IQ Test

Question:

James

A colleague told me about a recent CASP Memorandum saying that the NEPSY-II can be used as a replacement for an IQ test for determining eligibility for Specific Learning Disability (SLD). Is this true?

Tomas


Answer:

Hi Thomas,

I’m not aware of any recent CASP Memorandum regarding the NEPSY-II and African American assessment for determining eligibility for special education purposes.

The NEPSY-II cannot be used as a replacement for an IQ test because the NEPSY-II is not an IQ test.

Tests of cognitive ability or any other synonym for intelligence base their composite, domain, or broad factor scores, on statistically calculated latent variables inferred from performance on highly correlated subtests; e.g., two or more subtests provide you an inferred composite score for visual spatial reasoning.

An overall, global or full scale measure of cognitive ability is either a latent variable inferred from all subtests combined or a latent variable of latent variables, i.e., an inferred variable based on other inferred variables (composites, domains or broad factors). It is the reduction of an individual’s sum total of cognitive ability to a single number “a la” IQ that is what is objectionable in regards to Larry P.


The NEPSY II

The NEPSY II on the other hand, has subtests within each of its six domains that vary widely in terms of their presentation, how they are administered, how one responds and what is emphasized in scoring. “Therefore, subtests within a domain may not correlate highly with one another, and subtests across domains may correlate due to similar methodology and crossover of abilities. The domains are theoretically, not statistically, derived.” (Page 19 of the Clinical and Interpretive Manual) There are no latent variables in the NEPSY II for their domains, and there is obviously no overall NEPSY II score, because you are never to give the entire test. Please see March 2014’s question for further explanation and how the NEPSY II, Delis-Kaplan Executive Functioning System (D-KEFS) or other neuropsychological assessment tool might be useful in the assessment of African Americans for determining special education eligibility.
Therefore, if you question is “Can I use the NEPSY II as a replacement for an IQ test using the discrepancy model for SLD for African Americans?” The answer is NO.

If your question is, “Are we allowed to use the NEPSY II as part of a comprehensive evaluation that is addressing processing strengths and weaknesses (that are specifically tied to poor academic performance) to determine eligibility for SLD for African Americans?” then the answer is a qualified MAYBE. It would really depend on what is included in your comprehensive evaluation.

For more information on what the DCN views as a comprehensive assessment, please see Best Practice Guidelines for the Assessment of African American Students’ Cognitive Processes. Also, CASP in the fall of 2014 will large group discussion on this topic so keep your eye out for it.

Sincerely,

James


  • The Assessment of African American Students for Special Education

Question:

Hi James,

Given Larry P. what assessments do you use to assess cognitive functioning?  How can you get at it without standardized assessments?

Curious,
Jenny


Answer:

Hi Jenny,

At the DCN we use an assessment process we call The MATRIX. It is inclusive but not limited to the Best Practices from the 1994 CDE Memorandum and the 1989 Larry P. Task Force, and includes additional Best Practices used by Speech and Language Pathologists. This process is a trans-disciplinary approach and takes its information from:

  • Record Review
  • Interviews
  • Observation
  • Informal Assessment
  • Formal Standardized Testing for specific information in a processing area, but not one from a standardized cognitive ability/general ability/IQ battery, e.g., neuropsychological, social emotional concerns, adaptive behavior issues, and or developmental area.

Your question appears to be interested in what we do informally (instead of use standardized tests) so I will go into this a little more. In the context of The MATRIX, Informal Assessment includes a wide range of non- statistically normed, non-standardized activities which provide opportunities for a student to demonstrate various strengths and challenges. This process has its roots in both Dynamic and Authentic Assessment. The information derived from these activities complements data gathered through observations, interviews, work samples, and record reviews. Informal assessment may be used to gather general information about a student’s functioning or to try out hypotheses and clarify specific abilities. Informal assessment data often replace some data previously gathered through formal testing.
Informal Assessment can include elements from Observations and Interviews of the student as they include a wide variety of conversations and activities.

  • Activities may: 
    • Be unstructured or highly structured
    • Occur indoors or outside
    • Include 2 or more people

Instead of being dependent on national norms, which may not be reflective of the student, we rely on local norms that we can observe (classroom, grade level, or group level).

We do our best to incorporate preferred or familiar and unfamiliar games which address the following questions as part of our assessment:

    • What age range/development level is the game designed for?
    • How long did/does it take the student to learn the rules?
    • Can the student explain the rules to me?
    • Does the student just know the basics or is s/he able to employ strategies?
    • Compared to local norms, how does the student perform in with respect to the domains?

We begin to fill in the data as a Team as it comes in on The MATRIX (see example of a blank sheet below):

  • Plot the data on across each domain
  • Strengths -average or above average skills for the student’s peer group
  • Weakness - skills are noticeably below those of the student’s peer group
    • For children with Intellectual Disabilities, weaknesses across all or most domains would be we would expected
    • Relative strengths may be found in imitation, rote memory
  • Comments can include additional background information, low average skills, emerging skills, conflicting information. If we have conflicting information, further data collection is done to resolve it.

This will lead the team to be able to determine if there is a Specific Learning Disability through a Processing Strengths and Weaknesses Model. However, if you are using a discrepancy model, this will point to which processing areas are impacting achievement, given strengths in other areas.

The team can also compare Adaptive Behavior measures, and if two or more Adaptive Behavior measures are impacted, as well as a prevailing significant weakness across domains, ID would then be the appropriate special education disability category.

Thank you again for your question. I hope I addressed it to your satisfaction. If not, please ask me again and let me know where you’d like me to clarify or add more detail.

Sincerely,

James Hiramoto


REFERENCES:

Flanagan, D. P., & Harrison, P.L. (Eds.) (2012). Contemporary intellectual assessment, third edition: theories, tests, and issues, The Guilford Press.

Flanagan, D. P., & Harrison, P.L. (Eds.) (2005). Contemporary intellectual assessment, second edition: theories, tests, and issues, The Guilford Press.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Diagnostic Center North, California Department of Education.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (June & August 2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Presentation to Diagnostic Centers Central, South, and Fremont Unified, San Leandro, Twin Rivers and Mount Diablo Unified School Districts.

Valencia, R. R & Suzuki, L. A. (2001). Intelligence testing and minority students foundations, performance factors, and assessment issues. Sage Publications, Inc.


  • Standardized Tests normed for African Students

Question:

Hi James,

If IQ tests are not appropriately normed for African American students, how about all other standardized tests such as processing tests and achievement tests?

Nelson


Answer:

Hi Nelson,

Regarding Processing tests, CDE's prohibition is specifically for tests of intelligence and anything synonymous, e.g., tests of general cognitive ability, global ability index, etc. I would, however, look at the norming of the processing tests and be cautious in the use of them. I refer you to the DCN's website (www.dcn-cde.ca.gov) and the Analysis of Reliability and Validity sheet we at the DCN use to see the appropriateness of any assessment tool.

Regarding Achievement tests, again CDE's prohibition is specifically for tests of intelligence. Achievement tests measure what a child knows or has been directly taught through the education system. They are more a measure of what the child knows academically than of potential. They are also easily corroborated with information gathered from classrooms and home observations. However, it is not a good idea to use current nationally standardized achievement tests as the sole numeric point and contrast against cognitive ability as a means of determining eligibility for a specific learning disability using a discrepancy model. This is true for all students, not just those that are African American.

Currently, nationally normed achievement tests (those with norms collected prior to 2014-2015) have the fundamental problem of each state determining “what to teach and when to teach it” at every grade level. This will be the case until the fall of 2014, when most of the fifty states have Common Core State Standards up and running. Using current nationally standardized achievement tests to determine eligibility under the discrepancy model can under or overestimate the level of a discrepancy, depending upon if a state has set standards that are higher or lower than the nationally normed sample. Once the Common Core has been in place for a few years and achievement tests are re-normed, the information that achievement tests offer will change. Even so, it is prudent to place great stock on direct observations, work samples, interventions tried in the past and comments from classroom teacher and parents, regarding a student's level of performance, when determining eligibility.

Sincerely,

James Hiramoto


  • What to do with previously reported test scores for African American Special Education Students?

Question:

Hi James,

As far as I know, California, is the only state that does not allow IQ testing for Special Education for African Americans. What are we supposed to do with students who come from out of state that have this information in their Special Education Files?

Mary


Answer:

Dear Mary,

What if any of these kinds of test findings are found in a student’s file, from an outside agency or a different state?

The CDE issued a directive (Campbell, 1987):
“a qualified professional should identify appropriate data to be copied and purged of all IQ scores or references to information from IQ tests.”

The term purged has been interpreted as redacted (eliminating the reference by black pen making what is underneath unreadable Student v New Haven Unified 2007).

Once that is done, the IEP team must take into consideration all other information/findings (from interviews, observations, and other types of assessment both formal and informal) in the report, including its conclusion.

Even though the conclusion maybe heavily influenced by a test of general intelligence/cognitive ability this process isn’t as crazy as it sounds. The conclusion should be able to stand without the intelligence test based on all the other findings. If the conclusion cannot be made without that one test, they have violated the law where it states, “no single measure or assessment is used as the sole criterion for determining whether a pupil is an individual with exceptional needs or determining an appropriate educational program for the pupil” Article 2 CA Ed Code Sec. 56320 (e).

Thank you again for your question. I hope I addressed it to your satisfaction. If not, please ask me again and let me know where you’d like me to clarify or add more detail.

Sincerely,
James Hiramoto

REFERENCES:

---, (2012) Article 2 California Education Code Section. 56320 (e).

Flanagan, D. P., & Harrison, P.L. (Eds.) (2012). Contemporary intellectual assessment, third edition: theories, tests, and issues, The Guilford Press.

Flanagan, D. P., & Harrison, P.L. (Eds.) (2005). Contemporary intellectual assessment, second edition: theories, tests, and issues, The Guilford Press.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Diagnostic Center North, California Department of Education.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (June & August 2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Presentation to Diagnostic Centers Central, South, and Fremont Unified, San Leandro, Twin Rivers and Mount Diablo Unified School Districts.

Valencia, R. R & Suzuki, L. A. (2001). Intelligence testing and minority students foundations, performance factors, and assessment issues. Sage Publications, Inc.



  • The Assessment of African American Students for Special Education

Question:

Hello!

I work as a neuropsychologist and am seeking consultation about the assessment of African American (AA) children. If an AA child is suspected to fall in the intellectually disabled range, what assessments do you use to assess cognitive functioning (given that the use of IQ tests in not permitted under the Larry P. vs. Riles ruling)?

Thank you very much for your time,
Vivien


Answer:

Hello Vivien,

Your question is one that many are asking and is layered and complex. For the benefit of others I will start with the California Department of Education’s (CDE) 1994 Legal Advisory which states that districts should use in lieu of IQ tests, alternative means of assessment to determine identification and placement.

“Such techniques should include, and would not be limited to:

  • assessments of the pupil’s personal history and development
  • adaptive behavior
  • classroom performance
  • academic achievement
  • evaluative instruments designed to point out specific information relative to a pupil’s abilities and inabilities in specific skill areas”

Here are some alternative conceptual strategies suggested by the Larry P. Task Force (1989):

  • Developmental Assessment
  • Dynamic Assessment
  • Ecological Assessment
  • Information Processing
  • Neuropsychological Assessment
  • Psychological Processing
  • Skills Within Subjects

There is no current updated banned test list! I want to be very clear, there is no current banned test list!

The 1997 CDE Legal Memorandum states:

”No other list of tests has been recognized by the Department of Education for the purpose of finding school districts out of compliance in testing African-American students for special education…the original Larry P. decision was not limited to a specific set or sets of standardized intelligences tests, school districts should be advised that any standardized measure of intelligence should not be used with African-American students until such time as they are validated as unbiased by the State Board of Education and approved by the court.”

Here is where it gets tricky. Some districts have gone through the courts and have argued that some standardized tests of cognitive ability or general ability are OK to use with African American students for special education purposes because they do not measure “intelligence,” as if the absence of the word intelligence makes it any less so. More surprising is that they have won given comprehensive texts and even the authors themselves discussing their merits as a measure of intelligence.

What these districts fail to understand is that the 1997 CDE Legal Memorandum specifically states, “until such time as they are validated as unbiased by the State Board of Education and approved by the court.” To date the California State Board of Education has not validated any specific test of intelligence and therefore even though the courts may approve they are still not to be used. Silence on the part of CDE is not an endorsement of a test being validated or unbiased. Until such a time as CDE comes out and provides guidance on what can be used, the Riverside County SELPA-Guidelines for Assessing African-American Students offers the following:

“In making a determination of whether a test falls under the IQ test ban for African-American student one should consider:

  • Is the test standardized and does it purport to measure intelligence (cognition, mental ability or aptitude)?
  • Are the test results reported in the form of IQ or mental age?
  • Does evidence of the (construct) validity of the test rely on correlations with IQ tests?

An affirmative answer to any of these questions indicates that use of the test may fall within the ban (A Report of the Larry P. Task Force, 1989).

Thank you again for your question. I hope I addressed it to your satisfaction. If not, please ask me again and let me know where you’d like me to clarify or add more detail.

Sincerely,
James

REFERENCES:

California Department of Education, www.cde.ca.gov

Flanagan, D. P., & Harrison, P.L. (Eds.) (2012). Contemporary intellectual assessment, third edition: theories, tests, and issues, The Guilford Press.

Flanagan, D. P., & Harrison, P.L. (Eds.) (2005). Contemporary intellectual assessment, second edition: theories, tests, and issues, The Guilford Press.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Diagnostic Center North, California Department of Education.

Nielsen, M.A., Dawson, R. A., Friedland, M, Gallagher, S., M., Hiramoto, J. F., Jocic, N., & Stivers, M., (June & August 2012). Best practices guidelines for the assessment of African American students. Cognitive processes. Presentation to Diagnostic Centers Central, South, and Fremont Unified, San Leandro, Twin Rivers and Mount Diablo Unified School Districts.

Riverside County Special Education Local Plan Area (SELPA), Guidelines for Assessing African-American:
studentshttp://www.rcselpa.org/docs/policies/Section%20III%20Evaluations/III.h%20Guidelines%20for%20Assessing%20African-American%20Students.pdf

Valencia, R. R & Suzuki, L. A. (2001). Intelligence testing and minority students foundations, performance factors, and assessment issues. Sage Publications, Inc.