IRT readings

Getting Categorical CFA fit statistics with MPLUS

2009-05-11T08:22:00.000-07:00

CFA is a way to establish the dimensionalities of your data. For instance, unidimensionality and local independence are the assumptions of (unidimensional) IRT.

Just learned it today...

The default ML estimation does not provide model fit statistics for MPlus if you are running Categorical CFA. However if we change the method of estimation to wlsmv, it will spit out CFI, TLI and RMSEA in the output.

ANALYSIS: estimator=wlsmv;

Following are the links:

The Theory and Practice of Item Response Theory By R. J. de Ayala

2009-05-10T21:47:00.000-07:00

This is a nice book for IRT... explains a lot of stuffs pretty well in plain English... It is with information provided in this book did I figure out how to use item response pattern as the ID for bilog-MG output

The Theory and Practice of Item Response Theory By R. J. de Ayala

Chapter 10 IRT Applications: DIF, CAT and SCALE ANALYSIS (DIF only)

2009-05-10T21:29:00.001-07:00

Please find my notes for the DIF section at the following URL:

This chapter is in the book "Item Response Theory for Psychologists" by Embretson & Reise (2000)

http://sites.google.com/site/yaowensite/math-statistics-and-measurement-resources/item-response-theory/Chapter10IRTApplications.docx?attredirects=0

Item Response Theory by Baker

2009-04-14T11:53:00.000-07:00

Check out the free book by Baker "Item Response Theory." I think this is the first version of the IRT book by Baker and Kim...

Item Response Theory: Parameter Estimation Techniques, Baker & Kim (2004)

2009-04-14T11:52:00.000-07:00

Check out the book "Item Response Theory: Parameter Estimation Techniques, Baker & Kim (2004)" on google book...

http://books.google.com/books?id=y-Q_Q7pasJ0C&printsec=frontcover

Item Response Theory for Psychologists, Embretson & Reise (2000)

2009-04-14T11:51:00.000-07:00

Check out the book IRT for Psychologists on google book

http://books.google.com/books?id=rYU7rsi53gQC&printsec=frontcover&source=gbs_summary_r&cad=0

Chapter 3: Item Response Theory as Model-Based Measurement

2009-04-14T11:21:00.000-07:00

This chapter of Embretson & Reise spoke of issues such as the assumption of local independence. In addition, it also talks about ICC or Item Characteristic Curve....

Please see notes for more information...

Chapter 4: Binary IRT Model

2009-04-14T11:19:00.000-07:00

This chapter of Embretson & Reise discusses binary IRT models.... please see notes...

Chapter 3 Maximum Likelihood Estimation of Examinee Ability

2009-04-14T11:12:00.000-07:00

Since blogger doesn't have the capacity of showing math equations, I am linking this post to the word document I constructed while reading...

Notes

Logit for 3 PL

2009-02-28T06:28:00.000-08:00

From IRT readings

Chapter 2: The new rules of measurement

2009-02-04T13:41:00.001-08:00

Word document version

Rule 1: The standard error of measurement (SEM)

Old Rule 1: The standard error of measurement applies to all scores in a particular population
New Rule 1: The standard error of measurement differs across scores but generalizes across populations.

SEM for CTT is constant while that of IRT is variable.

CTT

Standard error of measurement for CTT:

SEM= (1-r_tt)^1/2σ

The estimated true score is derived from a linear transformation of raw score and the confidence interval also represented as straight lines.

IRT

The relationship between raw score and transformed score is non-linear. In addition, the confidence interval becomes wider at the extreme values.

Rule 2: Test length and reliability

Old Rule 2: Longer tests are more reliable than shorter tests
New Rule 2: Shorter tests can be more reliable than longer tests

CTT

Spearman-Brown prophecy formula:

Given r_ttis the reliability for the original test and n the number of parallel parts

r_nn=

An adaptive test, by nature fails to meet the assumptions because the test difficulties vary substantially.

Rule 3: Interchangeable Test Forms

Old Rule 3: Comparing test scores across multiple forms is optimal when test forms are parallel
New Rule 3: Comparing test scores across multiple forms is optimal when test difficulty levels vary between persons.

Gulliksen (1950) defined strict conditions for test parallelism in his exposition of CTT:

Equality of means and variances across test forms
Equality of covariance with external variables

Rule 4: Unbiased Assessment of Item Properties

Old Rule 4: Unbiased assessment of item properties depends on having representative samples
New Rule 4: Unbiased estimates of item properties may be obtained from unrepresentative samples

CTT

Item difficulty is p-value or the proportion of passing
Item discrimination is item-total correlation (e.g., biserial correlation)

Rule 5: Establishing Meaningful Scale Scores

Old Rule 5: Test scores obtain meaning by comparing their position in a norm group
New Rule 5: Test score obtain meaning by comparing their distance from items.

Rule 6: Establishing Scale Properties

Old rule 6: Interval scale properties are achieved by obtaining normal score distributions
New Rule 6: Interval scale properties are achieved by applying justifiable measurement models

Rule 7: Mixing Item Formats

Old Rule 7: Mixed item formats leads to unbalanced impact on test total scores.
New Rule 7: Mixed item formats can yield optimal test scores.

CTT

Z score

Rule 8: The Meaning of Change Scores

Old Rule 8: Change scores cannot be meaningfully compared when initial score levels differ.
New Rule 8: Change scores can be meaningfully compared when initial score levels differ.

X_J,Change=X_J2-X_J1

Beteiter (1963)- 3 fundamental problems with change scores:

Paradoxical reliabilities, such that the lower the pretest to posttest correlations, the higher the change score reliability
Spurious negative correlations between initial status and change (due to the subtraction
Different meaning from different initial levels

Rule 9: Factor Analysis of Binary Items

Old Rule 9: Factor analysis on binary items produces artifacts rather than factors
New Rule 9: Factor analysis on raw item data yields a full information factor analysis.

Phi correlation
Tetrachoric correlation
Full information factor analysis

Rule 10: Importance of Item Stimulus Features

Old Rule 10: Item stimulus features are unimportant compared to psychometric properties
New Rule 10: Item stimulus features can be directly related to psychometric properties.

This chapter should be in the end of the book.