38
Chapter 3: The Search For New Religious Orientations
And the Development of the Circumplex Religious Orientation Inventory (the CROI)
The previous chapter reviewed three of the most influential theories (models) of religious
orientation and highlighted five problematic issues concerning the measurement of these theories (models).
The problematic issues are as follows: 1) it is not very clear what the established scales (i.e., I, E, and Q)
are measuring; 2) it is not certain that the established scales are comprehensively measuring religious
orientation; 3) there are significant limits on the people that can validly complete the scales; 4) it is not
certain that scales are adequately reliable, and 5) most measures do not adequately control for response
sets. Resolving any of these issues may potentially have a large impact on the theories (models) the
measures represent (or the ability to evaluate them), but the resolution of the first two issues seem
especially prone to have an impact. Out of these two issues, it is the comprehensiveness of the measures
(and the models they represent) that is probably the easiest starting point for addressing the other issues, for
if another important religious orientation can be found, then the measures and models are clearly not
comprehensive and can be improved. Moreover, even if no new religious orientations are found, the
process of attempting to discover new orientations would shed more light on how well the existing models
represent the world and how well their measures capture the constructs represented in the models (i.e., the
meaning of the scales). In other words, the existing models and measures are much like the compass and
maps that a real-world explorer would use to search for new lands, and much like even an unsuccessful
voyage would help validate the explorers compass and maps, so even an unsuccessful search for new
religious orientation would lead to an increased understanding of the accuracy of the existing models and
measures.
With this in mind, a series of studies was launched whose principle purpose was to search for new
religious orientations. However, the study also had a series of other goals. The first and most important
goal was to develop a series of measures that had a very clear and specific interpretation. This would
further enable the identification of new religious orientations, while simultaneously clarifying the
interpretation of the existing measures.
The second goal was the development of a series of measures that would be usable with children,
nonreligious, uneducated, and older populations in any religion with a format that would further accurate
39
translations across languages. This would better enable the study of the development of religious
orientation and would also enable the study of the universal structure of religious orientation, assuming that
religious orientation is indeed similar across groups, cultures and religions (see Chapter 4).
The third goal was to improve the reliabilities of the original scales (especially the Ep and Es
scales), while keeping the scales short and economical. The fourth goal was to develop scales that can
better control for response sets. The achievement of both of these goals would help decrease the error
(noise) present in the measures, thereby allowing a more accurate determination of the relationships
between religious orientation and other variables of interest, such as prejudice, health, and personality.
Exploration and Scale Construction
Exploring for new religious orientations can be done in several ways. One logical place to start
would be to review and compare all the relevant theoretical models. However, there are several difficulties
with this approach. First, an in-depth review and comparison of all the relevant models is such a mammoth
task that is practically insurmountable (c.f., Wulff, 1999). Second, on a strictly theoretical level, it is
difficult to determine which theory is better and to determine which theory includes new religious
orientations. For example, a rather basic question such as whether Allports (1950) concept of mature
religion corresponds to his (1967) concept of intrinsic religion has gone unsolved despite a large amount of
debate on this topic (c.f., Hunt & King, 1973; Batson & Ventis, 1982). In short, while a theoretical review
is useful in proposing new religious orientation, the actual identification of new religious orientations is
best done in an empirical, statistical manner.
Yet, how can statistics determine whether a new religious orientation has been found? This can
be done in a couple of ways. Probably the easiest way is to find types of statements reflecting religious
orientation (e.g. the items of the I, E, and Q scales) that people answer in different ways. In other words,
statements of Type X and Type Y would reflect different religious orientations if responses to statements of
Type X were found to be at least partially independent of responses to statements of Type Y. As this is
probably the easiest way to find new religious orientations, this method, which is technically known as
factor analysis, formed the backbone of the early phases of the current investigation.
The second way to determine whether a new religious orientation is found, which is
complimentary to the first, is to examine the correlations of statements of Type X and Type Y with other
40
variables. If Type X is indeed separate from Type Y, then Type Xs and Type Ys correlations with other
variables should be different. This is typically called a validity analysis, and will be dealt with more in
Chapter 5.
An example of both of these procedures can be found in Kirkpatrick (1989). Kirkpatrick first
used factor analysis to show that the Extrinsic scale (Allport & Ross, 1967) was measuring two different
religious orientations: extrinsic-social (Es) and extrinsic-personal (Ep). Kirkpatrick then showed that Es
and Ep had different correlations with a number of variables, including the Intrinsic scale, prayer, and
church attendance. In other words, Kirkpatrick showed that the Es items were largely independent of the
Ep items and were differentially related to other variables of interest. As he used both factor analysis and
validity analysis, there could be little doubt about his results: that the Extrinsic scale was measuring two
largely independent religious orientations.
With this in mind, it was determined that a four-part process would be the best method to discover
new religious orientations. Part 1 would consist of constructing a large group of statements (an item pool)
regarding religious orientation. Part 2 would consist of factor analyzing the responses to the item pool and
identifying possible new religious orientations (factors) based on this analysis. Part 3 would consist of
readministering the item pool (or a section of it) to determine if the same religious orientations are found,
both within the culture of origin and in a comparison culture. Part 4 would consist of studying the
relationships between the new religious orientations and other variables of interest, such as the established
measures of religious orientation and the frequency of religious behaviors. Parts 1 to 3 are described in the
current chapter, while a description of part 4 can be found in Chapter 5.
Part 1: Item generation
The initial item pool was generated from the rewriting of the Gorsuch and McPherson (1989) Age
Universal I/E-Revised scales and Batsons (1991) Quest scale so that they could be meaningfully be
responded to by nonreligious participants. In addition, some items from the third edition of Leaks
Religious Maturity scale (RM3: Leak, 2000) were also modified in this manner, though large numbers of
items from this scale were not modified because it was perceived that their content and/or complex wording
seemed to somewhat preclude their use with nonreligious, young, old, or uneducated samples. For
example, the "my" in "my religion" was deleted in items such as "my whole approach to life is based on
41
my religion" to form items such as, "my whole approach to life is based on religion." Items originally
containing phrases like "religious beliefs" and " religious doubts," such as in the Quest item "it might be
said that I value my religious doubts and uncertainties," were changed to "beliefs about religion" and
"doubts about religion" to make items such as "it might be said that I value my doubts and uncertainties
about religion."
Next, questions were written that were perceived to be along similar themes as the original I, E, Q
and RM3 items. Last, after a review of the relevant empirical literature, questions were written to reflect
themes that were perceived as missing from the original I, E, Q, and RM3 scales. These items were written
mainly to reflect themes of interest, obligation, enjoyment of various religious activities, identification with
religious groups, superstition, material gain, fear, divine punishment, and self-criticism. Because the initial
samples that completed these items were Romanian, the items were translated from English to Romanian
and, to ensure that the translation was accurate, the items were then backtranslated from Romanian to
English by independent translators.
Part 2 Initial factor identification
The initial bank of 205 items was broken in three different sections based on what appeared to be
the most closely related themes in the items. These item banks of 69, 71 and 65 questions were then
completed by groups of 113, 112 and 117 participants respectively. All items were responded to on a fivepoint
format (1= strongly disagree, 2= disagree, 3= unsure, 4= agree, 5 = strongly agree). The results were
then analyzed using principal components analysis with promax and varimax rotation (Gorsuch, 1983).
Since this was an exploratory analysis, components (factors) were initially identified using a variety of
methods, namely the Kasier criterion and Cattell scree test. However, because the identification of factors
is relative to the size and composition of the item pool being analyzed, the overriding method of
identification at this early point was based on the perceived interpretability and meaningfulness of the
factors.
A small number of items (usually 6-10) were then grouped in preliminary scales and were the
only items included in successive administrations.1
These choices were based on the following, somewhat
conflicting, criteria: (1) content diversity, (2) high correlations with the other items in preliminary version
of the scale and high factor loadings, (3) low correlations with other scales and low loadings on other
42
factors, (4) stability of the items factor loadings and correlations, (5) inclusion of at least 33% reversed
scored items on each scale, (6) the retention of an alpha of at least .75 for the 6-item version of each scale,
(7) readability and simplicity of vocabulary, (8) inclusion of only the items that seemed to be able to be
meaningfully and predictably responded to by people of all religions and degrees of religiosity (including
nonreligious respondents).
Each successive administration included additional items that were seen as related, though
conceptually more diverse, in an effort to increase the content diversity of the scales. Additional items were
also written and included in an effort to fix perceived deficiencies in the scales (e.g., a possible lack of
reverse scored items) or in the comprehensiveness of the inventory as a whole. Over 200 additional items
were written in this fashion, which means that approximately 400 items were analyzed overall.
This process was then repeated five times with samples of 150, 133, 129, 108, and 129 completing
questionnaires of 104, 106, 100, 104, and 151 items until each of the identified factors were believed to
satisfactorily meet the previously mentioned criteria. However, each administration did not include every
scale item pool, but only those being improved. Most questionnaires contained approximately six scale
item pools. The preliminary scales from this eighth Romanian analysis were then completed by an
American sample (n=113) and the items for each scale were selected.2
Participants in the Scale Construction Studies
Overall, a total of 1064 participants were involved in the scale construction process. Of these 951 were
Romanian (599 females, 361 males) and 113 were American (77 females, 34 males). The Romanian
participants were students at either a large state university or a smaller, private university in Bucharest,
Romania. The vast majority of Romanians classified themselves as Romanian Orthodox (n=850), though
some participants were Catholics (n=20) and some had no religious affiliation (n=62).3
The measures were
voluntarily completed in class, at their homes, or on the spot in the hallways of the universities.
The American participants were students at a small, Methodist affiliated liberal arts university in
the Midwest who complete the scales in return for partial course credit. Most of these participants were
protestant (n=69), though some were Catholic (n=26), and a few had no religious affiliation (n=14).
43
Participants in the Replication and Validity Studies
Findings from thirteen replication and validity samples will be discussed through the rest of this
book. These thirteen samples were comprised of a total of 2331 participants (655 Americans, and 1676
Romanians). Twelve of the samples are college students or dormitory residents, and one sample, RO 1, is
comprised of hospital employees and their friends and family. As the books aim was not to describe
populations, but to relate the religious orientations to other measures within the same sample, the use of
primarily college samples seems somewhat justifiable. Yet, the books reliance on university samples
leaves open the possibility that the current findings will not replicate in more representative samples drawn
from the American or Romanian populations. However, as will be shown later, the findings discussed in
this book are typically robust across the Romanian and American samples, and thus it seems doubtful that
more representative samples drawn from each culture would diverge more from the college samples that
the US college samples do from the Romanian college samples.
Sections of Romanian sample 4, and US samples 3 and 4 completed additional questionnaires.
These subsamples are indicated by the addition of letters after the sample number, such as Romanian
sample 4b and US sample 4a.
Unless otherwise mentioned, all measures were completed using a five-point format ranging from
"strongly disagree" to "strongly agree." The sample number, nation, number of participants, sex and a
brief description of each sample are shown in Table 3.1. However, not everyone indicated their sex and as
a result the number of known males of females does not always equal the total number of participants in a
sample, which is especially clear in RO 4.The sample numbers shown in Table 3.1 are used throughout the
book.
44
Table 3.1: Data Sources and Brief Description of Samples
Sample
Name Nation Sample Size Males Females Mean Age Sample Description
RO 1 Romania 84 23 61 32.86
Bucharest hospital employees and their
family and friends
RO 2 Romania 129 24 105 22.37 University Students in Bucharest
RO 3 Romania 143 77 66 21.93 Dormitory residents in Bucharest
RO 4 Romania 291 85 151 21.43 Dormitory residents in Bucharest
RO 4a Romania 104 14 39 21.47 Dormitory residents in Bucharest
RO 4b Romania 142 29 62 21.43 Dormitory residents in Bucharest
RO 5 Romania 186 112 72 21.70 Dormitory residents in Bucharest
RO 6 Romania 200 105 93 21.94 Dormitory residents in Bucharest
RO 7 Romania 223 141 80 22.35 Dormitory residents in Bucharest
RO 8 Romania 189 89 100 22.10 Dormitory residents in Bucharest
RO 9 Romania 231 105 122 22.25 Dormitory residents in Bucharest
US 1 US 96 32 61 18.6 University students in Tennessee
US 2 US 113 34 77 19.03 University students in Kansas
US 3 US 189 80 108 19.79 University students in Tennessee
US 3a US 137 54 82 19.85 University students in Tennessee
US 4 US 257 95 154 19.00 University students in Tennessee
US 4a US 141 50 88 19.00 University students in Tennessee
Religious Orientations
Eight different religious orientations were found that are believed to either underlie (in some
sense) the I, E, Q and RM scales or seemed to offer a more comprehensive picture of how people approach
religion. In all, ten scales were developed to measure 8 religious orientations and 4 suborientations. The
following is a short description of each orientation based on the analysis of the types of items (including
those that did not make it onto the final version of the scale) that consistently loaded on these dimensions
across different the factor analyses done on the eight different Romanian samples.4
Together these
measures form the Circumplex Religious Orientation Inventory, or CROI for short. The revised version of
the CROI, which is the version recommended for use, is included in the appendix.
45
Orientation 1: Religiosity (committed, unreflective religion)
This orientation consists of two suborientations: centrality and personal and can be measured by
averaging the Centrality and Personal scales (Centrality + Personal / 2). This orientation is the amount to
which religion is a central, meaning endowing, comforting aspect of an individuals life. In essence, this
orientation exists to the degree to which a personal is devoted to a loving, comforting, personal God.
Table 3.2: Descriptive statistics and reliability of the Religiosity scale
Sample Reversed items Item mean SD Reliability
RO 1 6 of 12 3.51 .60 .87
RO 3 6 of 12 3.11 .74 .88
RO 4 6 of 12 3.39 .74 .91
RO 5 6 of 12 3.45 .86 .93
RO 6 6 of 12 3.60 .75 .90
RO 7 6 of 12 3.47 .86 .91
RO 8 6 of 12 3.68 .77 .92
RO 9 6 of 12 3.42 .75 .90
US 1 6 of 12 3.98 .68 .88
US 2 6 of 12 3.55 .82 .93
US 3 6 of 12 2.75 .72 .90
US 4 6 of 12 2.64 .78 .91
Mean 6 of 12 3.38 .76 .90
Suborientation 1: Centrality
This orientation is the degree to which religion is important and central in a persons life. In this
sense, it is very similar to what Allport (1950) and Leak (2000) called religion as "master-motive." Yet,
this orientation also was composed of items that seemed to be measuring the "comprehensive," or meaning
endowing, aspect in Allports (1950) theory of mature religion. In other words, religion is an important,
central and meaning endowing aspect of life for people high in this orientation. It is thus designed to
measure what people normally mean by "religious" or "committed."
However, despite numerous parallels between this orientation and Allports thought, this
orientation should probably be seen as neo-Allportian. This is because of the fact that it is applicable to the
46
nonreligious and is not truly conceptually modeled after his Intrinsic or mature religion, for it is unclear
where the nonreligious might be able to fit into Allports bipolar conception of religion.
Items were written such that a nonreligious person was intended to score lower on this orientation than
a more religious person. Thus, this process has enabled theoretical points to be suggested for the
identification of uncommitted (sometimes referred to as "nonreligious" participants) and extremely
committed participants. These suggested points are as follows: a uncommitted individual has an average
item mean of less than two on the scale (i.e., total score on the scale divided by the number of items is less
than two) and an extremely committed individual has an average item mean of four or greater (i.e., total
score on the scale divided by the number of items is greater than or equal to four).
Table 3.3: The Centrality scale
1. The meaning I give my life comes from religion.
2. Religion is the driving force in my life.
3. I find the purpose of my life in religion
4. (-)Religion is NOT the most important thing in my life.
5. (-)There are many things in my life that are more important than religion.
6. (-)Religion is NOT a big part of my life.
47
Table 3.4: Descriptive statistics and reliability of the Centrality scale
Sample Reversed items Item mean SD Reliability
RO 1 3 of 6 3.12 .78 .88
RO 2 3 of 6 2.50 1.00 .92
RO 3 3 of 6 2.82 .93 .90
RO 4 3 of 6 2.51 .90 .89
RO 5 3 of 6 3.24 1.02 .93
RO 6 3 of 6 3.38 .96 .91
RO 7 3 of 6 3.26 .95 .86
RO 8 3 of 6 3.47 .90 .91
RO 9 3 of 6 3.19 .94 .90
US 1 3 of 6 3.78 .92 .91
US 2 3 of 6 3.17 1.07 .94
US 3 3 of 6 3.51 .90 .90
US 4 3 of 6 2.43 .92 .89
Mean 3 of 6 3.11 .94 .90
Suborientation 2: Personal
The Personal religious orientation is an offshoot of the original extrinsic personal orientation
(Gorsuch & McPherson, 1989; Kirkpatrick, 1989; Leong & Zachar; Rice 1993). Like the original version,
the core of this suborientation is an approach towards religion in order to gain comfort, protection,
forgiveness, and help in general. This suborientation also seems to reflect the idea of being especially
loved and cared for by God. It should be noted that this approach is actively encouraged by many, if not
most, denominations (including Romanian Orthodox) and is thus quite socially acceptable.
The placement of this suborientation in the same category as Centrality may seem debatable for
two major reasons. First, Personal is designed to measure aspects of religion that are often believed to be
indicative of extrinsic, bad religion in Allports (1950) thought. Centrality, on the other hand, is designed
to measure many of the aspects that Allport (1950) designated as part of intrinsic, good religion. Second,
the original orientations of Extrinsic-Personal and Intrinsic religion were thought to be orthogonal (e.g.,
Gorsuch & McPherson, 1989:Kirkpatrick 1989: Rice 1993) and thus unrelated.
48
Unlike Allport (1950), it was the authors observation that comfort, protection, devotion and
commitment are "means and ends" that are inextricably intertwined (Pargament, 1992). Though the exact
causes of this relationship are not yet clear, some possibilities, such as self esteem (Benson & Spilka, 1973)
and terror management (Solomon, Greenberg & Pyszezynski, 1991), have been suggested that would
explain the relationship between Centrality and Personal, as well as the otherwise surprising lack of
religions based on a hating, unloving, distant, and punitive god.
The reason that Ep and I were originally believed to be orthogonal seems to be that the original
Extrinsic-Personal items were heavily value laidenned (e.g., "the primary purpose of prayer is to gain relief
and protection"). The items in the new Personal scale are not (e.g., "God comforts me and protects me
from harm"), which therefore allows people to more freely endorse or reject both. In fact, both Kennedy
and Gorsuch (1989) and Kirkpatrick and Hood (1990) have also reported some evidence that shows that the
orientations are positively related to one another even when the value-laidenned measures of Ep and I are
used. In short, Personal and Centrality are both part of Religiosity because they seem to reflect almost
universally integrated "means and ends" of religious life (Pargament, 1992).
Table 3.5: The Personal scale
1. (-)Prayer is NOT a very good way to seek guidance.
2. God comforts and shelters me.
3. (-)God might watch me, but he does NOT help me.
4. God protects me if I pray.
5. God helps me if I ask him.
6. (-)I do NOT turn to God more when I have problems.
49
Table 3.6: Descriptive statistics and reliability of the Personal scale
Sample Reversed items Item mean SD Reliability
RO 1 3 of 6 3.91 .58 .74
RO 2 3 of 6 3.50 .92 .87
RO 3 3 of 6 3.39 .72 .75
RO 4 3 of 6 2.98 .69 .84
RO 5 3 of 6 3.70 .80 .83
RO 6 3 of 6 3.82 .67 .74
RO 7 3 of 6 3.69 .89 .83
RO 8 3 of 6 3.90 .75 .83
RO 9 3 of 6 3.66 .72 .78
US 1 3 of 6 4.17 .61 .74
US 2 3 of 6 3.93 .69 .83
US 3 3 of 6 3.98 .69 .80
US 4 3 of 6 2.86 .77 .83
Mean 3 of 6 3.65 .73 .80
Orientation 2: Rewards and Punishments (committed and unreflective religion)
The Rewards and Punishments orientation (RP) consists of the degree to which an individual
approaches religion as a method of gaining rewards or avoiding punishment. Alternatively, this orientation
can be conceptualized as the degree to which a person believes that God is actively helping those "on his
side" to achieve their goals and desires, while at the same time hindering the goals and desires of those less
fully in his good graces. For someone high in this orientation, God is thought to reward and punish in
direct response to their beliefs, requests and behavior. Therefore, God is believed to be controllable and
the individuals beliefs, requests and behavior create direct results. In other words, the rewards and
punishments orientation is designed to measure what Berrenberg (1986) referred to as God mediated,
internal control and what Furnham (1982) referred to as an essential part of fundamentalist belief systems.
Although the RP orientation was not modeled after any particular system of thought, this
orientation has many obvious parallels in the theories of Kohlberg (1981) and Oser and Gmunder (1991).
Yet, many differences also exist between the RP orientation and these theories. For instance, the RP
orientation (like all the orientations presented) is seen as existing on a continuum in every individual and
50
not as some type of developmental stage or special type of religion. Instead, this orientation is believed,
much like belief in superstition and astrology, to be the result of efforts to increase perceived control over
the environment. Yet, unlike belief in superstition or astrology, the rewards and punishments orientation is
designed to measure an active, though mediated, form of perceived control over the environment.
The RP orientation can be measured through the averaging of a respondents scores on the Gain
and Punishment scales. The original RP scale is derived from averaging the original Gain and Punishment
scales. The revised RP scale is derived from combining the revised Gain and Punishment scales, which are
more reliable and have better construct validity.
Table 3.7: Descriptive statistics and reliability of the original Rewards and Punishment scale
Sample Version Reversed items Item mean SD Reliability
RO 1 Original 3 of 11 3.30 .50 ---
RO 3 Original 3 of 11 2.77 .62 ---
RO 4 Original 3 of 11 3.01 .65 ---
US 1 Original 3 of 11 3.05 .67 ---
US 2 Original 3 of 11 2.52 .57 ---
US 3 Original 3 of 11 1.76 .55 ---
Mean Original 3 of 11 2.74 .59 ---
Table 3.8: Descriptive statistics and reliability of the revised Rewards and Punishments scale
Sample Version Reversed items Item mean SD Reliability
RO 3 Revised 3 of 12 2.61 .62 .86
RO 4a Revised 3 of 12 2.79 .70 .88
RO 5 Revised 3 of 12 2.74 .66 .86
RO 6 Revised 3 of 12 2.90 .64 .86
RO 7 Revised 3 of 12 2.90 .71 .83
RO 8 Revised 3 of 12 3.02 .66 .86
RO 9 Revised 3 of 12 2.83 .59 .82
US 3 Revised 3 of 12 1.61 .55 .78
US 4 Revised 3 of 12 1.69 .61 .83
Mean Revised 3 of 12 2.57 .64 .84
51
Suborientation 3: Gain
This suborientation consists of the degree to which religion is approached as a method for gaining
wealth, health, success, and other personal desires. This dimension was originally thought to be measuring
how much people use their religion to aid themselves materially through the increased status and contacts
that religion may sometimes provide. However, it soon became apparent that divine aid laid at the heart of
this dimension. In essence, the Gain suborientation consists of a persons propensity toward modern rain
dances and other methods of gaining divine assistance. Thus, the Gain suborientation is obviously related
to the comforting, protecting elements of Personal, but is much more extreme, materialistic, and less
socially acceptable in most cultures. It is perhaps this last difference, the social acceptability of the type of
divine aid requested, that forms the true distinction between Ep and Gain, though this possibility has yet to
be examined empirically.
Table 3.9: The original Gain scale
1. If I become more faithful, God would improve my health.
2. Praying to God is a good way to help my career.
3. If I am faithful, God will help me be successful in life.
4. (-) God does NOT reward the faithful with improved health.
5. Prayer is a good way to get what I want.
52
Table 3.10: Descriptive statistics and reliability of the original Gain scale
Sample Version Reversed items Item mean SD Reliability
RO 1 Original 1 of 5 3.42 .55 .63
RO 3 Original 1 of 5 2.82 .77 .81
RO 4 Original 1 of 5 1.75 .69 .78
RO 5 Original 1 of 5 3.03 .84 .80
RO 6 Original 1 of 5 3.17 .73 .73
RO 7 Original 1 of 5 3.12 .84 .72
RO 8 Original 1 of 5 3.30 .77 .75
RO 9 Original 1 of 5 3.06 .76 .75
US 1 Original 1 of 5 3.15 .82 .76
US 2 Original 1 of 5 2.65 .68 .75
US 3 Original 1 of 5 1.75 .69 .65
US 4 Original 1 of 5 1.90 .76 .72
Mean Original 1 of 5 2.76 .74 .74
Table 3.11: The revised Gain scale
1. If I become more faithful, God would improve my health.
2. Praying to God is a good way to help my career.
3. If I am faithful, God will help me be successful in life.
4. (-) God does NOT reward the faithful with improved health.
5. Prayer is a good way to get what I want.
6. (-) God would not improve my career if I became more faithful.
53
Table 3.12: Descriptive statistics and reliability of the revised Gain scale
Sample Version Reversed items Item mean SD Reliability
RO 3 Revised 2 of 6 2.83 .72 .79
RO 4a Revised 2 of 6 2.72 .83 .84
RO 5 Revised 2 of 6 3.03 .82 .83
RO 6 Revised 2 of 6 3.15 .71 .76
RO 7 Revised 2 of 6 3.09 .77 .72
RO 8 Revised 2 of 6 3.26 .74 .78
RO 9 Revised 2 of 6 3.05 .73 .78
US 3 Revised 2 of 6 2.79 .65 .71
US 4 Revised 2 of 6 1.94 .71 .73
Mean Revised 2 of 6 2.87 .74 .77
Suborientaion 4: Punishment
The core of this suborientation is an approach towards religion that is heavily colored by a fear of
God, by conscious attempts to avoid divine punishment, and a belief that negative events are controllable as
well as meaningful. This might also be perceived as an empirical measure of the common complaint that
religion is a form of "death insurance." However, this dimension goes farther than this and also includes
avoidance of divine punishment in life.
Table 3.13: The original Punishment scale
1. I have obligations to God that if NOT respected will cause bad things to happen to me.
2. Bad things happen in life to those who do NOT worship God.
3. (-)I do NOT believe in divine punishment for incorrect behavior.
4. (-)Im NOT often scared about the possibility of God punishing me for incorrect behavior.
5. Im scared that if I would NOT go to church/synagogue God would cause something bad to
happen.
6. People should fear Gods anger and punishment
54
Table 3.14: Descriptive statistics and reliability of the original Punishment scale
Sample Version Reversed items Item mean SD Reliability
RO 1 Original 2 of 6 3.19 .57 .65
RO 2 Original 2 of 6 2.29 .83 .82
RO 3 Original 2 of 6 2.73 .61 .67
RO 4 Original 2 of 6 1.76 .60 .71
US 1 Original 2 of 6 2.95 .70 .66
US 2 Original 2 of 6 2.39 .64 .66
US 3 Original 2 of 6 1.76 .60 .54
Mean Original 2 of 6 2.44 .65 .67
Table 3.15: The revised Punishment scale
1. I have obligations to God that if NOT respected will cause bad things to happen to me.
2. Bad things happen in life to those who do NOT worship God.
3. Im scared that if I would NOT go to church/synagogue God would cause something bad to
happen.
4. God would cause bad things to happen to me if I became less faithful.
5. If I dont do certain things, God will cause bad things to happen to me.
6. (-) Making fun of religion will not affect your health.
55
Table 3.16: Descriptive statistics and reliability of the revised Punishment scale
Sample Version Reversed items Item mean SD Reliability
RO 3 Revised 1 of 6 2.39 .68 .79
RO 4a Revised 1 of 6 1.69 .81 .82
RO 5 Revised 1 of 6 2.46 .67 .76
RO 6 Revised 1 of 6 2.66 .75 .82
RO 7 Revised 1 of 6 2.71 .84 .79
RO 8 Revised 1 of 6 2.79 .72 .80
RO 9 Revised 1 of 6 2.60 .61 .70
US 3 Revised 1 of 6 2.41 .61 .68
US 4 Revised 1 of 6 1.44 .65 .77
Mean Revised 1 of 6 2.35 .70 .77
Orientation 3: Extrinsic Social (uncommitted and unreflective religion)
The Extrinsic Social orientation (Social) exists to the degree to which a person involves
themselves in religion in an effort to make or see friends and other social acquaintances. More specifically,
this orientation consists of going to religious services and functions (i.e., church attendance) in an effort to
make or see friends. It should be noted that of all the scales of the CROI presented in this chapter, the
Social scale is undoubtedly the most single-minded in purpose and unity in item content. The scale
presented in this article is not alone in this regard, as every scale made to tap this factor has had the same
problem. This, in part, explains why it has normally formed a separate factor while simultaneously
existing in a theoretical void. In fact, all efforts to increase the diversity of item content of this scale, and
thereby elaborate on the dimension, failed in this investigation.
56
Table 3.17: The Social scale
1. If I go to church/synagogue it is to make friends.
2. (-) I do NOT go to church/synagogue to make friends.
3. Going to church/synagogue is very important because I can spend time with friends.
4. If I go to church/synagogue it is because I enjoy seeing people I know there.
5. If I go to church/synagogue it is to make and see friends.
6. (-) If I go to church/synagogue it is NOT to see my friends.
Note. The term "church/synagogue" was used because the Romanian language has difficulty
expressing the more neutral phrase "a place of worship." In languages that can express the phrase "a
place of worship," such as English can, it may be beneficial to use it in lieu of the phrase
"church/synagogue" in areas where three or more organized religions are expected to be found. The term
"worships services" is not recommended in cultures where people attend their places of worship for reasons
other than organized religious services, such as for personal or group prayer, remembrance of the dead, or
social gatherings.
Table 3.18: Descriptive statistics and reliability of the Social scale
Sample Reversed items Item mean SD Reliability
RO 1 2 of 6 2.18 .54 .75
RO 3 2 of 6 1.82 .56 .73
RO 4 2 of 6 1.01 .69 .70
RO 5 2 of 6 2.01 .61 .72
RO 6 2 of 6 2.05 .74 .82
RO 7 2 of 6 2.19 .89 .84
RO 8 2 of 6 1.97 .65 .78
RO 9 2 of 6 2.15 .70 .82
US 1 2 of 6 2.12 .69 .81
US 2 2 of 6 2.01 .69 .86
US 3 2 of 6 2.01 .69 .85
US 4 2 of 6 1.09 .73 .84
Mean 2 of 6 1.88 .68 .79
57
Orientation 4: Obligation (uncommitted and unreflective religion)
This orientation consists of the amount to which a person feels social pressure to act or be
religious. In other words, religious behavior is a form of duty or obligation for someone high in this
orientation. This is another orientation that is outside of the Allportian tradition. However, this dimension
closely resembles the obligation factor identified by Echemendia and Pargament (1982) and might have
important implications for the development of the other orientations. Namely, it is possible that Obligation
is a form of impediment toward the development of intrinsically motivated forms of religion (e.g.,
Religiosity or Interest), for an individuals locus of control is shifted externally when obligation is
perceived (Deci & Ryan, 1985). This would, of course, have important implications for how religious
values should best be passed on to the younger generations.
Table 3.19: The original Obligation scale
1. I feel a lot of pressure from my friends and family to go to religious services.
2. My friends and family would be upset if I did NOT go to church/synagogue.
3. I feel pressured because the important people in my life place more importance on being religious
than I do.
4. My friends and family place much more importance than I do on going to church/synagogue.
5. (-) Nobody pressures me into being religious.
6. (-) Nobody important in my life would be angry with me if they thought I never went to
church/synagogue.
58
Table 3.20: Descriptive statistics and reliability of the original Obligation scale
Sample Version Reversed items Item mean SD Reliability
RO 1 Original 2 of 6 2.55 .55 .58
RO 2 Original 2 of 6 2.02 .77 .78
RO 3 Original 2 of 6 2.27 .59 .66
RO 4 Original 2 of 6 1.61 .78 .61
RO 5 Original 2 of 6 2.30 .54 .59
RO 6 Original 2 of 6 2.34 .57 .57
RO 7 Original 2 of 6 2.58 .74 .65
RO 8 Original 2 of 6 2.36 .52 .54
RO 9 Original 2 of 6 2.40 .59 .62
US 1 Original 2 of 6 2.66 .76 .70
US 2 Original 2 of 6 2.49 .85 .79
US 3 Original 2 of 6 1.61 .78 .72
US 4 Original 2 of 6 1.71 .74 .61
Mean Original 2 of 6 2.22 .68 .65
Table 3.21: The revised Obligation scale
1. I feel a lot of pressure from my friends and family to go to religious services.
2. My friends and family would be upset if I did NOT go to church/synagogue.
3. I feel pressured because the important people in my life place more importance on being
religious than I do.
4. My friends and family place much more importance than I do on going to church/synagogue.
5. (-) Nobody pressures me into being religious.
6. (-) Nobody important in my life would be angry with me if they thought I never went to
church/synagogue.
7. (-) I dont feel pressure to go to church/synagogue because important people in my life go.
8. (-) Important people in my life do not influence whether I go to church/synagogue.
59
Table 3.22: Descriptive statistics and reliability of the revised Obligation scale
Sample Version Reversed items Item mean SD Reliability
RO 4a Revised 4 of 8 2.25 .58 .71
RO 5 Revised 4 of 8 2.32 .51 .62
RO 6 Revised 4 of 8 2.40 .55 .61
RO 7 Revised 4 of 8 2.59 .69 .69
RO 8 Revised 4 of 8 2.39 .52 .62
RO 9 Revised 4 of 8 2.40 .56 .67
US 3 Revised 4 of 8 2.61 .77 .79
US 4 Revised 4 of 8 1.72 .71 .76
Mean Revised 4 of 8 2.34 .61 .68
Orientation 5: Doubt (uncommitted and reflective religion)
Doubt is the first of the four orientations that were developed out of the Quest scale and Batsons
presentation of the Quest concept. The Doubt orientation exists to the degree to which an individual enjoys
and values their religious doubts, uncertainties and questions. It is thus, a type of intrinsic motivation,
though toward doubting religion instead of towards religion itself. Doubt therefore seems to be a partial
embodiment of the concerns of Donahue (1985), who believed that Quest was related to agnosticism and to
a tendency toward "reflexively respond[ing] ,,why to every answer given" (p. 413).
Watson, Morris and Hoods (1989a) research is also supportive of the current view of Doubt.
These researchers also identified a component of the six-item Q scale (which they also termed Doubt) that
was composed of two of the same items that were used to initially identify the Doubt factor, one of which is
still part of the current Doubt scale (item number 5). These researchers found that their Doubt factor was
negatively correlated with the Intrinsic scale and was generally maladaptive.
60
Table 3.23: The Doubt scale
1. It can be good to doubt your beliefs about religion.
2. It does NOT bother me when I have doubts about my beliefs about religion.
3. (-) It is better to be sure about your religious beliefs than have some doubts.
4. (-) I do NOT like to question my beliefs about religion.
5. I value my doubts and uncertainties about religion.
6. (-) It bothers me to question my beliefs about religion.
7. For me, doubting is an important part of what it means to be religious.
Table 3.24: Descriptive statistics and reliability of the Doubt scale
Sample Reversed items Item mean SD Reliability
RO 1 3 of 7 3.03 .62 .59
RO 2 3 of 7 3.33 .77 .80
RO 3 3 of 7 3.04 .66 .75
RO 4 3 of 7 2.01 .77 .79
RO 5 3 of 7 2.78 .72 .79
RO 6 3 of 7 2.70 .65 .75
RO 7 3 of 7 2.86 .72 .70
RO 8 3 of 7 2.63 .74 .84
RO 9 3 of 7 2.80 .68 .79
US 1 3 of 7 2.67 .60 .68
US 2 3 of 7 3.27 .76 .84
US 3 3 of 7 3.01 .77 .81
US 4 3 of 7 1.95 .65 .73
Mean 3 of 7 2.78 .70 .76
Orientation 6: Tentativeness (uncommitted and reflective religion)
The Tentative orientation is the degree to which an individual is self-critical and uncertain of the
objective validity of their beliefs about religion (whether religious or nonreligious). This dimension was
originally developed to measure the self-criticism that is seemingly missing from Batsonss hypothesized
Self Criticism and Perception of Doubt as Positive component of Quest. In doing so, Tentativeness also
61
seems to decently capture the "opened-ended" and "resisting clear-cut, pat answers" (Batson et al., 1993, p.
169) parts of the Q construct, and also seems to represent to some extent the opposite of Batsons
conception of the I scale as a measure of true belief. This orientation is also very closely related to the
Heuristic Quality factor of Leaks RM3 scale (Leak, 2000). The main difference between these two factors
is that Leaks Heuristic Quality factor seems to assume people have religious beliefs and can therefore act
wholeheartedly on them, while Tentativeness makes no such claim. Tentativeness, much like Doubt, also
would seem to be somewhat related to agnosticism, for the truth about religion does not seem ever to be
knowable for someone high in this orientation.
Table 3.25: The Tentativeness scale
1. You can never know the complete truth about religious matters.
2. You can never be sure if your beliefs about religion are correct.
3. (-) Its easy to know whether my beliefs about religion are correct
4. (-) Im sure my beliefs about religion are correct.
5. Some of my beliefs about religions are probably wrong.
6. (-) It is obvious that my beliefs about God are correct.
62
Table 3.26: Descriptive statistics and reliability of the Tentativeness scale
Sample Reversed items Item mean SD Reliability
RO 1 2 of 6 3.03 .62 .78
RO 2 2 of 6 3.82 .69 .80
RO 3 2 of 6 3.30 .62 .71
RO 4 2 of 6 1.93 .78 .75
RO 5 2 of 6 3.04 .76 .81
RO 6 2 of 6 2.96 .66 .71
RO 7 2 of 6 2.88 .65 .58
RO 8 2 of 6 3.03 .71 .78
RO 9 2 of 6 3.07 .71 .77
US 1 2 of 6 2.83 .68 .71
US 2 2 of 6 3.38 .78 .83
Us 3 2 of 6 2.93 .78 .78
US 4 2 of 6 1.97 .75 .77
Mean 2 of 6 2.94 .71 .75
Orientation 7: Dialog (uncommitted and reflective religion)
This orientation is the degree to which an individuals beliefs about religion are in a dialog with
their experiences. In other words, Dialog is the degree to which an individual is aware that their religion is
affected by "the contradictions and tragedies of life" (Batson et al., 1993 p. 169) as well as other life
experiences. It is therefore seen as part of the original Q component of Readiness to Face Existential
Questions Without Reducing Their Complexity. Yet, Batsons original component seems to have
combined Interest with Dialog.
63
Table 3.27: The Dialog scale
1. I have reexamined my beliefs about religion when my life has changed.
2. (-) My experiences have NOT changed my feelings toward religion.
3. (-) My beliefs about religion did NOT change because of major events in my life.
4. (-) Personal tragedies and hard times in my life have NOT changed how I think about religion.
5. My life experiences have made me reexamine my views on religion.
6. (-) No event in my life changed how I think about religion.
Table 3.28: Descriptive statistics and reliability of the Dialog scale
Sample Reversed items Item mean SD Reliability
RO 1 4 of 6 2.77 .60 .63
RO 2 4 of 6 2.86 1.08 .88
RO 3 4 of 6 2.65 1.01 .81
RO 4 4 of 6 2.31 .92 .82
RO 5 4 of 6 2.62 .82 .82
RO 6 4 of 6 2.62 .80 .80
RO 7 4 of 6 2.70 .56 .77
RO 8 4 of 6 2.62 .78 .79
RO 9 4 of 6 2.68 .77 .80
US 1 4 of 6 3.22 .86 .81
US 2 4 of 6 3.53 .92 .89
US 3 4 of 6 3.31 .92 .84
US 4 4 of 6 2.12 .78 .79
Mean 4 of 6 2.77 .83 .80
Orientation 8: Interest (committed and reflective religion)
The Interest orientation consists of the amount to which an individual enjoys learning, reading and
talking about religion and religious concepts. In addition, this orientation also includes an enjoyment of
studying and analyzing religious concepts and theories. It is thus a form of intrinsic motivation that bridges
the distinction between the religious doubter and the believer. It is a form of doubt that is often found
operating within religion and religious traditions, though an individual high in this dimension might not
64
even perceive themselves as having doubts or uncertainties about their religious beliefs. Instead, they
believe they "analyze" their beliefs. Yet, it should be noted that nonreligious individuals, like many
scholars and researchers, can also be high in this dimension.
The Interest orientation has many similarities with Allports concept of religiously mature doubt
as well as Batsons concept of Q. In fact, Interest is believed to be a combination of the interest in religion
inherent in the original I scale and the "readiness to face existential questions without reducing their
complexity" (Batson & Schoenrade, 1991, p. 431) component of the Q scale. However, Interest should not
be seen as a version of Q, for Interest does not necessarily entail the open-ended, somewhat agnostic search
that is seemingly at the center of the Q construct. These agnostic aspects of Quest are better represented by
Doubt, Dialog and Tentativeness. Yet, it is possible that Batson (1976) was thinking of something
resembling the Interest orientation when he questionably used the Hebrew prophets as examples of
individuals high in his Q orientation, for endless searching and questioning was not at the heart of their
message (Donahue, 1985).
It is also supportive of the current view of Interest that Watson, Morris and Hood (1989a) found a
similar factor existing in the six-item Q scale, which they termed Identity. This factor was composed of
one of the three Q items that originally allowed for the identification of Interest as part of the Q construct.5
Table 3.29: The Interest scale
1. (-)Im NOT very curious about religious theories.
2. I like to closely examine religious ideas.
3. I find religious discussions fascinating.
4. (-)I am NOT interested in theoretical discussions about religion.
5. I love to find out new things about religion
6. (-)I do NOT like to learn about religion.
65
Table 3.30: Descriptive statistics and reliability of the Interest scale
Sample Reversed items Item mean SD Reliability
RO 1 3 of 6 3.37 .67 .78
RO 2 3 of 6 3.28 .90 .89
RO 3 3 of 6 3.38 .75 .82
RO 4 3 of 6 2.70 .77 .84
RO 5 3 of 6 3.47 .73 .81
RO 6 3 of 6 3.54 .73 .81
RO 7 3 of 6 3.40 .91 .85
RO 8 3 of 6 3.59 .70 .83
RO 9 3 of 6 3.50 .66 .75
US 1 3 of 6 3.78 .70 .82
US 2 3 of 6 3.87 .76 .89
US 3 3 of 6 3.70 .77 .83
US 4 3 of 6 2.63 .70 .81
Mean 3 of 6 3.40 .75 .83
Psychometric Properties of the CROI
Ten new scales measuring eight different orientations and four suborientations were developed in
Romania and America through the use of theoretically guided, factor analytic procedures. Yet, two
important questions still remain to be answered before the validity of these measures can be addressed.
The first of these regards the reliability of the scales. The second question regards whether the scales of the
CROI are indeed independent dimensions of religious orientation, as determined by factor analysis.
Reliability
The first question that must be addressed is whether the proposed CROI orientations can be
measured reliably both within a culture and across different cultures. As reliability coefficients (coefficient
alpha) are measures of the amount of error (noise) present in scales, a low reliability will make it more
difficult to determine the true relationship between two variables. In short, reliability coefficients are a
measure of scale quality, and the higher the reliability coefficients the better.
In addition, there are some rules of thumb that help to interpret reliability coefficients (also called
alpha coefficients). Reliability coefficients below .60 are generally classified as unacceptable, though
66
measures with reliabilities below .60 are still occasionally used in research, especially when the scales are
very short (2-4 items). Reliability coefficients between .60 and .70 are typically classified as poor.
Reliability coefficients of .70 to .80 are classified as acceptable for use with research. Reliability
coefficients over .80 are typically classified as good, and coefficients over .90 are classified as excellent.
Table 3.31 reviews the reliability coefficients (also called alpha coefficients) reported in thirteen
separate administrations (nine Romanian and four American) of the 60-item original CROI and the 63-item
revised CROI. As can be seen, eight of the original scales and nine of the revised scales have adequate to
good levels of reliability in both America and Romania. Centrality, Interest, and Dialog appear to be the
most reliable scale of the CROI, with all of these scale having good levels of reliability in both cultures.
However, the original Punishment and Obligation scales have relatively poor reliability, with the revised
versions being noticeably more reliable than the originals. Yet, the revised Obligation has a relatively poor
level of reliability, though only in Romania. In addition, the scales appear to be slightly more reliable in
the American samples than they are in the Romanian sample, which is surprising given that the measures
were developed primarily in Romania. Overall, the scales of the CROI appear to be adequately reliable for
use either separately or as an inventory.
67
Table 3.31: Reliability of the CROI
Sample
1 3 4 5 6 7 8 9 1 2 3 4
US
Mean
Reliability
RO
Mean
Reliability
Overall
Mean
Reliability
Nation RO RO RO RO RO RO RO RO US US US US US RO ---
Religiosity .87 .88 .91 .93 .90 .91 .92 .90 .88 .93 .90 .91 .91 .90 .90
Personal .74 .75 .84 .83 .74 .83 .83 .78 .74 .83 .80 .83 .80 .79 .80
Centrality .88 .90 .89 .93 .91 .86 .91 .90 .91 .94 .90 .89 .91 .90 .90
Rewards and Punishments --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
Rewards and Punishments-R --- .86 .88 .86 .86 .83 .86 .82 --- --- .78 .83 .81 .85 .84
Gain .63 .81 .78 .80 .73 .72 .75 .75 .76 .75 .65 .72 .72 .75 .74
Gain-R --- .79 .84 .83 .76 .72 .78 .78 --- --- .71 .73 .72 .79 .77
Punishment .65 .67 .71 --- --- --- --- --- .66 .66 .54 .78 .66 .68 .67
Punishment-R --- .79 .82 .76 .82 .79 .80 .78 --- --- .68 .77 .73 .79 .78
Social .75 .73 .70 .72 .82 .84 .78 .82 .81 .86 .85 .84 .84 .77 .79
Obligation .58 .66 .61 .59 .57 .65 .54 .62 .70 .79 .72 .61 .71 .60 .64
Obligation-R --- --- .71 .62 .61 .69 .62 .67 --- --- .79 .76 .78 .65 .68
Doubt .59 .75 .79 .79 .75 .70 .84 .79 .68 .84 .81 .73 .77 .75 .76
Tentativeness .78 .71 .75 .81 .71 .58 .78 .77 .71 .83 .78 .77 .77 .74 .75
Dialog .63 .81 .82 .92 .80 .77 .79 .8 .81 .89 .84 .79 .83 .79 .81
Interest .78 .82 .84 .81 .81 .85 .83 .75 .82 .89 .83 .81 .84 .81 .82
MEAN-Original CROI .70 .76 .77 --- --- --- --- --- .76 .83 .77 .78 .78 .76 .77
MEAN-Revised CROI --- --- .80 .80 .77 .76 .80 .78 --- --- .80 .79 .80 .78 .79
68
Factor Structure
The second question that has been left unanswered is whether the factor structure of the CROI is
stable across samples and cultures. In other words, it is not clear whether the factor structure of the CROI
is the same either in samples draw from a single culture or in those drawn from different cultures.
Factorial stability is extremely important for any inventory, with Everett (1988) logically suggesting that
the only factors worthy of note are those that are replicable. This is especially true in samples drawn from
the same culture, for if the structure is different from sample to sample, this would suggest that the
dimensions are either not empirically definable or that the scales are not adequately tapping the dimensions
they are intended to tap. In either case, a lack of factorial stability within a cultural group would pose a
large problem.
On the other hand, cultural differences in the factorial structure of inventories are so frequently
found they are often even expected. Yet, despite this fact, a lack of factor stability across cultures is still
quite problematic, for it suggests that the inventory may not be able to be interpreted in the same manner in
both cultures. For example, it is possible that certain religious orientation dimensions could be culturally
specific, and do not even exist in some cultures, for there is a precedent for this in mainstream psychology,
with even some supposedly universal traits, such as those of personality, being known to be at least
somewhat culturally dependent.
Factorial Stability
With the importance of factor stability in mind, the average congruence coefficients, or Tuckers
phi (1951), for the first three Romanian samples that completed the original version of the CROI are shown
in Figure 3.1, while the first three American samples are shown in Figure 3.2. These congruence
coefficients were computed from principle components analyses using orthogonal (varimax) rotation. In
addition, orthogonal procrustean rotation was used in order to eliminate superficial differences in the factor
structure of different samples, with the two small samples from each culture being rotated to the
coordinates of the largest sample of that culture.7
As the structure of the CROI has not yet been proven, a
variety of different numbers of factors (components) were extracted and their stability compared. This
procedure should allow for the determination of the number of factors that best represent the inventory in
both Romania and America, with only those factors that replicate well being kept (Everett, 1988).
69
Figure3.1: Factor stability across three Romanian samples
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Components Extracted
Congruence
Figure 3.2: Factor stability across three American samples
0
0.2
0.4
0.6
0.8
1
1 3 5 7 9 11 13 15
Components Extracted
Congruence
As can be seen in both figures 3.1 and 3.2, there is a sudden drop in factor replicability after eight
factors are extracted. This trend is present in all three Romanian and in all three American samples. Thus,
in both America and Romania, the CROI seems to be best represented by at most eight factors. However,
70
this finding does not imply that the factor structure is the same across both cultures. To study this
possibility, the factorial structure of the RO 4 sample was compared with the US 3 sample. The results of
this comparison can be seen in Figure 3.3
Figure 3.3: Factor congruence across American and Romanian samples
0
0.2
0.4
0.6
0.8
1
1 3 5 7 9 11 13 15
Components Extracted
Congruence
As can be seen, the same trend that was found in both the Romanian and American cultures was
also present when the Romanian and American facture structures are compared, with a significant drop in
congruence occurring after eight factor are extracted.8
This confirms that the CROI is best represented by
at most eight factors.
However, it should be noted that only individual factors with a congruence coefficient of .90 or
higher are generally considered to be invariant across samples. As most all factors of the CROI have
coefficients below .90, practically none of the factors can be classified as invariant in either culture or
between cultures. So while the factor structures of all the samples are similar to one another, they are not
so similar as to be classified as invariant. This lack of invariance could be caused by many things, such as
careless responding by the participants, some faulty items, poorly constructed scales (see the section on
reliability), as well as actual differences in the structure of religious orientation across the samples.
71
Factor Interpretation
The factor structure of the CROI has just been shown to be similar across cultures and samples,
but this similarity does not mean that the observed factors correspond to the theorized dimensions, or even
how many factors are important. To examine these questions further, the factor structure of the largest
Romanian and American samples were examined in more detail. First, as the congruence coefficients
suggested that at most eight factors are replicable, the eight-factor solution was first examined. However,
the eighth factor of the eight-factor solution had only two significant loadings, which are defined as
loadings over .4. In both sample these two loadings were made by Obligation item 6 and Punishment item
3. Because this factor had only two significant loadings, it thus appeared to be a trivial factor and was
discarded.
As only the first seven factors of the eight-factor solution appeared to be meaningful, the sevenfactor
solution was also examined. The seven-factor solution appeared to be nearly identical to the first
seven factors of the eight-factor solution, with the average level of congruence between the first seven
factors of the eight-factor solution and the respective factors in the seven-factor solution being .99 in the
US sample and .95 in the Romanian one. Because of this high level of similarity between the two
solutions and the lack of any trivial factors in the seven-factor solution, the seven-factor solution appeared
to be the more appropriate one.
To help in interpreting the factors, Table 3.32 shows the seven-factor solution for both the
American and Romanian samples, along with the levels of congruence of each factor. Judging by the
items with significant loadings on each of the factors, which are indicated in bold, a tentative interpretation
of them can be made.
72
Table 3.32: The seven-factor solution of the original CROI in Romania and the United States
Factors 1 2 3 4 5 6 7
Nation US RO US RO US RO US RO US RO US RO US RO
Personal 1 .61 .40 -.10 -.16 .07 .09 -.06 -.09 .29 .20 .17 .39 -.18 -.19
Personal 2 .57 .47 -.16 -.08 .01 .15 -.13 -.06 .39 .00 .24 .51 -.14 -.08
Personal 3 .54 .46 -.17 -.01 .17 .01 -.03 -.21 .35 .15 .24 .32 -.21 -.22
Personal 4 .11 .49 -.39 -.24 .05 .03 .20 -.14 .11 .21 .51 .60 -.27 -.18
Personal 5 .46 .44 -.24 -.26 .13 .06 -.10 -.04 .20 .14 .33 .49 -.17 -.27
Personal 6 .45 .51 -.11 -.02 .17 .15 .02 -.22 .18 -.03 .07 .26 -.08 -.10
Centrality 1 .78 .61 -.08 -.20 -.03 .08 -.02 -.06 .13 .24 .03 .22 -.03 .07
Centrality 2 .80 .72 .02 -.12 -.05 .07 -.16 .04 .09 .18 .17 .15 -.08 .04
Centrality 3 .76 .67 -.03 -.15 -.05 .08 -.12 -.04 .17 .18 .16 .26 -.13 .10
Centrality 4 .77 .58 -.03 -.20 -.04 .10 -.17 -.01 .15 .14 -.06 .22 -.13 .03
Centrality 5 .71 .72 -.08 -.06 .06 .13 -.15 -.03 .19 .09 -.10 .11 .01 .05
Centrality 6 .79 .73 .05 -.01 -.05 .07 -.13 -.15 .18 .13 -.05 .13 .06 .02
Gain 1 .00 .17 -.27 -.19 .08 .04 .09 .17 .04 .13 .58 .59 -.04 .08
Gain 2 .27 .34 -.14 -.20 .06 .10 .04 .18 -.03 .06 .55 .60 -.07 -.12
Gain 3 .40 .31 -.17 -.18 .13 -.07 -.16 .13 -.07 .12 .47 .67 -.15 -.05
Gain 4 .22 .20 -.42 -.19 .28 .08 -.08 -.06 -.08 .20 .34 .38 -.16 -.08
Gain 5 -.16 .30 .10 -.16 -.07 .11 .17 .01 -.08 .05 .57 .51 .17 .03
Punishment 1 .10 .34 -.06 -.05 .10 .08 -.01 -.06 -.05 .05 .55 .41 .15 .35
Punishment 2 .22 .26 -.25 -.28 -.03 .04 -.13 .02 -.06 .05 .50 .48 .21 .27
Punishment 3 .22 .47 -.10 .13 .15 .11 -.20 -.29 -.11 -.09 .25 .03 .13 .06
Punishment 4 -.05 .28 -.13 -.06 .22 .26 -.17 -.13 .04 .09 .34 .36 .07 .16
Punishment 5 -.04 .11 .11 -.02 -.28 .08 .26 -.01 -.01 -.12 .40 .40 .36 .50
Punishment 6 .37 .50 -.15 -.12 -.05 -.01 -.01 -.16 .02 -.06 .06 .23 .27 .21
Interest 1 .10 .34 .29 .18 .05 .04 -.10 .00 .63 .52 -.03 -.30 -.09 .06
Interest 2 .23 .22 .19 .06 .25 .06 -.07 -.10 .54 .74 -.18 -.14 .07 .18
Interest 3 .26 .25 -.04 .23 .06 .06 -.12 -.01 .72 .69 -.09 -.14 -.09 .01
Interest 4 .18 .23 .05 .14 .13 .08 -.13 -.04 .67 .73 -.02 -.14 -.01 -.01
Interest 5 .14 .14 .19 -.04 -.08 .12 -.11 -.07 .71 .79 .15 .05 -.06 .11
Interest 6 .30 .26 .06 -.06 .15 .06 -.19 -.09 .59 .63 -.15 .05 -.04 .00
Social 1 -.20 .10 -.03 -.04 -.04 .09 .62 .69 -.07 -.08 .00 -.06 .03 -.02
Social 2 -.01 .00 .00 .10 .18 .06 .70 .49 -.29 -.11 -.16 -.04 -.02 .18
Social 3 .11 .07 .13 .08 .02 .07 .74 .68 -.10 -.06 .15 .20 -.07 -.02
Social 4 -.14 .24 .10 .12 .03 -.05 .70 .61 -.08 .02 .15 -.02 .17 .08
Social 5 -.21 .03 .02 .16 .01 .06 .77 .68 -.06 -.23 .03 -.09 .05 .13
Social 6 -.09 -.15 .06 -.12 .16 -.05 .78 .37 -.11 .00 -.11 .07 -.01 .19
Obligation 1 -.25 -.17 .03 -.13 .13 -.04 .04 .03 -.10 .05 .17 .31 .70 .59
Obligation 2 .34 .19 -.07 -.05 .18 -.02 .05 .22 -.15 -.02 -.05 .24 .56 .44
Obligation 3 -.35 -.06 -.04 -.13 .26 -.12 .02 .16 .12 .04 -.05 .32 .60 .41
Obligation 4 -.30 -.01 -.03 .08 .09 -.07 .16 .03 .13 .03 .04 .13 .56 .50
Obligation 5 -.18 -.02 .07 -.14 .28 .10 .05 .21 -.21 -.01 .04 -.01 .58 .54
Obligation 6 .31 .40 .02 .05 .08 .01 -.22 -.04 -.10 -.25 .10 .00 .53 .40
Doubt 1 -.20 -.20 .67 .64 .10 .05 .12 .03 .18 .03 -.19 -.08 .13 .03
Doubt 2 -.17 -.39 .62 .46 -.06 -.03 .06 .12 -.06 .05 -.14 -.04 -.04 -.06
73
Doubt 3 -.28 -.14 .60 .56 .11 .21 .11 -.14 .05 .00 -.05 -.16 -.10 .11
Doubt 4 .04 -.34 .66 .53 .15 .11 .07 .07 .07 .15 -.07 -.07 -.14 -.09
Doubt 5 -.16 -.21 .52 .54 .20 -.03 -.05 .14 .17 .19 -.09 .02 .12 -.11
Doubt 6 .16 -.28 .54 .55 .09 -.01 -.03 .07 -.04 .04 -.17 -.03 -.29 -.18
Doubt 7 -.02 -.25 .71 .46 .09 -.04 .05 .15 .14 .12 -.12 .09 .02 .03
Tentativeness 1 -.26 -.14 .47 .41 -.13 -.05 .09 .01 .35 .07 .02 -.15 .17 -.07
Tentativeness 2 -.45 -.36 .55 .45 -.10 .03 -.03 .16 -.04 .05 .01 .16 .20 .13
Tentativeness 3 -.51 -.29 .46 .44 .04 .06 -.01 -.20 .16 .07 -.17 .00 .04 -.09
Tentativeness 4 -.57 -.26 .34 .61 .16 .17 -.07 -.21 .02 -.14 .05 .02 -.03 -.15
Tentativeness 5 -.40 -.45 .08 .22 .28 .11 .01 .03 .25 .19 -.03 .24 .19 -.15
Tentativeness 6 -.53 -.34 .31 .56 .21 .21 .03 -.09 .01 -.02 -.09 -.07 -.11 -.12
Dialog 1 .02 .15 .28 -.01 .44 .51 .14 .08 .17 .28 .05 .17 .12 -.16
Dialog 2 .03 -.16 -.03 -.04 .67 .73 .09 -.14 .04 .13 -.06 -.09 .03 -.01
Dialog 3 -.04 -.07 -.01 .14 .75 .72 .08 -.04 .00 -.04 .19 -.06 .01 .08
Dialog 4 -.04 -.11 .07 -.04 .79 .70 -.03 .08 .01 .15 .08 -.04 .09 .11
Dialog 5 -.01 .07 .18 -.06 .70 .69 .08 -.05 .14 .22 .07 .01 .25 .00
Dialog 6 -.07 -.08 .18 .04 .75 .83 -.02 -.01 .11 .09 .10 -.01 .20 .03
Congruence .86 .87 .81 .81 .87 .82 .83
Note: Loadings in bold as significant. Loadings that are underlined are unpredicted, significant loadings.
Loadings that are double underlined are not significant, but were predicted to be significant.
As can be seen in Table 3.32, almost all the factors are defined exclusively by items from only one
scale. Factor one appears to be a Religiosity factor, with 11 of the 12 items making up the Religiosity scale
(which is comprised of the Personal and Centrality scales) loading significantly on the factor in the US
sample, and all 12 items loading significantly on it in the Romanian sample. However, factor 1 also could
be interpreted as a "true belief" factor, an interpretation that Batson (Batson and Ventis, 1982; Batson et al,
1993) has argued that the similar Intrinsic scale should also have. This interpretation is made possible
because 5 of 6 of the Tentativeness items loaded negatively on this factor in the US sample. However, only
one of the Tentativeness items loaded negatively on this factor in the Romanian sample, suggesting the
heart of this factor is Religiosity, and not true belief.
Factor two appears to be primarily a Doubt factor, for all of the Doubt items loaded on the factor
in both cultures. However, Tentativeness is also intimately involved with this factor, with 3 of 6 of the
Tentativeness items loading on it in the US sample and 5 of the 6 items loading on it in the Romanian
sample. If this finding is replicated in future samples, it would suggest that the Tentativeness and Doubt
74
scales are part of a higher level religious questioning factor, which is logical considering that people who
actively question religion should be more tentative about their beliefs and vice versa.
Most of the rest of the factors have a clear interpretation. Factor three appears to be a Dialog
factor, as all Dialog item load on it in both cultures. Factor four is clearly a Social factor, with only Social
items significantly loading on it. Factor five is an Interest factor, with all the Interest items loading on it in
both cultures. Factor seven is clearly an Obligation factor, with all the Obligation items having significant
loadings on it in both cultures.
However, factor six is a little more complicated, but it appears to be a Rewards and Punishments
Factor, for 4 of the 5 Gain items loaded significantly on the factor in both cultures, as did all three of the
Punishment items that were retained on the Revised Punishment scale. The other three items from the
original Punishment scale did not load on that factor, which, in concert with the reliability data already
presented, suggests that Revised Punishment scale is superior to the original one. In addition, some of the
Personal items also had significant loadings on this factor, suggesting that the Personal dimension is closely
linked to this factor as well.
Overall, the factor structure of the inventory is pretty much as predicted and is very similar in both
samples. The only major unpredicted finding was that the Tentativeness scale did not form a factor of its
own, but blurred across the religiosity and doubt factors. Other than those caused by the Tentativeness
scale, there were only 3 unpredicted significant loadings in the American sample, and 7 in the Romanian
sample. As for predicted significant loadings that were not actually significant, there were 4 in the
American sample and 4 in the Romanian sample, not including those caused by the Tentativeness scale not
forming an independent factor.
Yet, the congruence coefficients for the factors ranged from .87 to .81, which is slightly below the
.90 that is generally regarded as indicating perfect congruence. Thus, the congruence coefficients indicate
that all of the factors contain some non-negligible differences across cultures. More specifically, factors 1
and 2 are two of the most replicable factors judging by the congruence coefficients. However, as
previously discussed, a visual examination of the significant factor loadings in the two sample indicate that
factors 1 and 2 are the most divergent across cultures, and that factors 3 to 7 are virtually identical across
cultures. These conflicting results suggest that it is primarily the non-significant factor loadings that are
75
different across cultures, which may indicate slight differences in the correlations between the scales in the
two cultures.
In summary, despite the fact that the congruence coefficients tell us that the factors contain nonnegligible
differences across cultures, an examination of the significant loadings suggests that the factors
are actually very similar across cultures. This difference in interpretation may have arisen because
congruence coefficients are affected by all factor loadings, including those that are non-significant and nonpredicted.
Thus, as can be seen on factors 3, 4, 5, and 7, only items that are predicted to significantly load
on a factor may indeed do so, but because of the 54 other non-predicted, non-significant loadings, the
congruence coefficients indicated that the factors are not invariant. In fact, judging by the congruence
coefficients, factors 3, 4 and 7 are the least replicable factors of the CROI, while judging from the
significant loadings, these three factors are some of the cleanest, most replicable factors of the CROI.
Confirmatory Factor Analysis
A more advanced way of examining the factor structure of the CROI is through the process of
confirmatory factor analysis (CFA), which is a specific type of structure equation modeling. CFA is quite
different from exploratory factor analysis, because in CFA the investigator tells the program which items
load on each factor and which factors correlate with each other. The program then tells you how much the
data fit your expectations of them through a large variety of goodness of fit measures. To use an analogy
of a child trying to figure out which of the differently shaped pegs go in each hole, CFA tells you how
difficult it is to fit the pegs (data) in the hole (structure) you expected it to go into. On the other hand,
exploratory factor analysis would represents something like the computers best guess of how many holes
(factors) there are and which pegs (items) should go in each.
However, CFA is also much more advanced than exploratory factor analysis in the types of
questions that it can answer. For example, CFA can tell you whether there are response biases present in
the data, and whether they are masking smaller hypothesized factors. When multiple groups are examined
using CFA, it can tell you whether correlations between factors are of different sizes.
To examine the factor structure of the CROI in both the U.S. and Romania, US sample 4 was used
and Romania samples 5 and 6 were combined to make one large Romanian sample. Thus, unlike in the
76
exploratory analysis previously reported, which used the 60-item original CROI, the CFA was used to
study the structure of the 63-item, revised CROI.
To try to determine what is the ideal structure of the CROI in the US and Romania, six different
models were compared using the US and Romanian samples separately. The first model directly tested the
findings of the exploratory factor analysis and predicted 8 orthogonal factors. In the second model, method
factors were added to try to account for the error caused by response sets. In the third model, 8 oblique
factors were predicted, and response-set based, method factors were added in the fourth model. The fifth
and sixth models were pretty much the same as the fourth model, though in the fifth model, the rewards and
punishment factor that was present in the four previous models was split into the Gain and Punishment
suborientations. Building on the fifth model, the sixth model split the Religiosity orientation into the
Centrality and Personal suborientations, and thus had 10 oblique factors and 2 method factors.
The results of these analyses are shown in Table 3.33. As can be seen, the 2
and RMSEA become
smaller in each subsequent model and the CFI and PCFI become larger in each subsequent model, which
indicates that the sixth model is the best one. In other words, assuming 10 oblique factors and two
response-set method factors fit the data the best in both the US and Romania. Though this model was the
best fitting model, it is not yet clear that the model fit the data very well. To examine this question, we
must look at the actual levels of the four goodness of fit statistics (2
, CFI, PCFI, and RMSEA) for each
sample. The interpretation of each goodness of fit statistic varies somewhat from author to author, though
it is generally accepted that: a 2
/df ratio under 2 indicates good fit; a CFI over .90 indicates good fit; a
parsimony adjusted CFI (PCFI) over .50 indicates good fit, and that a RMSEA under .06 indicates good fit
(Byrne, 2001). Using these standards, it would appear that Model 6 fits the data well in both samples
across all goodness of fit measures except the CFI. However, the parsimony adjusted CFI (PCFI) is
considered to be a superior version of the CFI (Byrne, 2001), for CFI does not adequately adjust for model
simplicity. In short, it appears that Model 6 fits the data well in both the US and Romania, and thus that the
CROI is best represented by 10 oblique factors and two method factors.
The previous analyses examined the factor structure of the CROI separately in the US and
Romania and thus, though we know which Model 6 is the best fitting model in both countries, we do not
yet know whether the factor structure is invariant across cultures. In CFA, differences in fit tend to come
77
from differences in factor loadings across samples, differences in factor correlations (covariances) across
samples, and differences in factor variances across samples. To test for invariance across factors in CFA,
all three of these areas are first required to be equal, and then the 2
of this model is compared with when
they are allowed to vary. When this is done using the CROI, the 2
was 7197 with 3796 degrees of freedom
when the factor loadings, covariances, and factor variances were constrained equal. When this is compared
with when everything was allowed to vary, the 2
for each sample showed a difference of 359 and 110
degrees of freedom, which is a statistically significant difference. Using follow up analyses, it became
apparent that there were some differences between the factor loadings, covariances and factor means of
each sample.
As there does seem to be some differences across the samples in all three important areas,
subsequent analyses were conducted to determine exactly where the differences occurred. Following
Byrne (2001), if something was found to be invariant across samples, it was added to the baseline model
that each new constraint would be tested against.
First, the factors loadings of each factor were constrained separately to determine which factors
loadings were different across samples. When this was done, it became apparent that some of the factor
loadings of Dialog, Gain, Tentativeness, and Social were different across groups.
Next, the covariances (correlations) between factors were constrained separately to determine
which were different across samples. When this was done, it became apparent that Centrality and Personal
were more negatively correlated with Punishment and Interest in Romania, and more positively correlated
with Tentativeness, Social, and Obligation in the US. Gain was more negatively correlated with Doubt,
Tentativeness, and Obligation in the US, and more positively correlated with Interest in Romania.
Punishment was more negatively correlated with Interest in the US and more positively correlated with
Dialog in Romania. Interest was more negatively correlated with Obligation and Social in the US and it
was more negatively correlated with Doubt in Romania. Doubt was more positively related to
Tentativeness and Dialog in the US. Lastly, Social was more positively correlated with Obligation in
Romania.
78
Last, the variances of each factor were examined to determine whether they were different across
samples. When this was done, it became apparent that Punishment had a higher variance in Romania, and
that Obligation had a higher variance in the US.
Overall, the CFA results indicated that the CROI is best represented by 10 oblique, content factors
and two method factors, and that this model fits the data well in both Romania and America. However,
much like the exploratory results, the CFA results showed that there were differences in the factor structure
across the cultures.
It may seem counterintuitive at first that this model can fit the data well in Romania and America,
but the inventory has different factor structures in these two countries. However, this pattern of findings is
logical, for it has been shown that the areas where the model is good and the areas where the model is bad
are different in the two samples. This is a bit like findings that a shirt fits two men relatively well, but it is
slightly short in the arms on one and slightly long in the arms on the other. Thus, though the shirt fits both
men relatively well, it fits them both differently.
Overall, the exploratory and confirmatory factor analyses suggest that the factor structure is quite
close to the predicted one in both cultures, with confirmatory factor analysis revealing that each of the 10
scales of the CROI are tapping into 10 separable religious orientations in both Romania and America.
Because of this, the higher order factors of Religiosity and Rewards and Punishments will not be
extensively dealt with in the rest of the book. In addition, both the exploratory and confirmatory factor
analyses suggest that the factor structure of the CROI is similar in both cultures, but it is not the same. As a
result, the scales of the CROI can validly be used within each culture, but the means of each should not be
compared across cultures, for the scales behave differently enough in each culture to make comparisons of
means across cultures suspect.
As the 10 scales of the CROI have now been shown to be measuring separate religious
orientations in both Romania and America, we can now turn to more interesting questions, such as the
structure of religious orientation (Chapter 4) and its correlates (Chapter 5).
79
Table 3.33: Goodness of fit statistics for six models of the revised CROI
Model
Model Name US RomaniaNumber
df 2
CFI PCFI RMSEA 2
CFI PCFI RMSEA
1 Orthogonal 1890 4480 .619 .599 .073 5326 .663 .642 .069
2 Orthogonal with method factors 1888 4039 .684 .661 .067 4764 .718 .694 .063
3 Oblique 1862 3966 .691 .658 .066 4506 .741 .706 .061
4 Oblique with method Factors 1860 3496 .759 .723 .059 3984 .792 .754 .054
5 Oblique with method,
gain and punishment separate 1852 3392 .774 .734 .057 3727 .816 .774 .051
6 Oblique with method,
all suborientations separate 1843 3255 .792 .748 .055 3581 .830 .783 .049
80