An overall purpose of factor analysis is to summarize data so that relationships and patterns can be easily interpreted and understood. It is normally used to regroup variables into a limited set of clusters based on shared variance. Hence, it helps to isolate constructs and concepts.
According to Child, 2006 factor analysis uses mathematical procedures for the overview of interrelated measures to determine patterns in a set of variables. From Harman, 1976 the process of discovering a simple method of interpretation of observed data is known as parsimony, and this is essentially the aim of factor analysis.
The main branches of factor analysis are Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). According to Child, CFA attempts to confirm hypotheses and uses path analysis diagrams to represent variables and factors, whereas EFA tries to uncover complex patterns by exploring the dataset and testing predictions.
Factor analysis works on the conception that measurable and observable variables can be reduced to fewer latent variables that share a common variance and are unobservable, which is known as reducing dimensionality (Bartholomew et al., 2011).
We carryout factor analysis because large datasets that consist of several variables can be reduced by observing ‘groups’ of variables called factors.
According to Child, 2006 to carry out a factor analysis, there has to be univariate and multivariate normality within the data. According to Field, 2009 it is also important that there is an absence of univariate and multivariate outliers. Also, a determining factor is based on the assumption that there is a linear relationship between the factors and the variables when computing the correlations (Gorsuch, 1983).
There are many ways to express the ideas behind factor analysis theoretically but we are going to treat the mathematical and geometrical approach.
P denotes the number of variables (X1, X2,…,Xp) and m denotes the number of underlying factors (F1, F2,…,Fm) for classical factor analysis for mathematical model. Xj is the variable represented in latent factors. This model assumes that there are m underlying factors whereby each observed variables is a linear function of these factors together with a residual variate. This model seeks to obtain maximum correlation of data
Where j = 1, 2………, p, and aj1 and aj2 are the factor loading
Factor analysis uses matrix algebra when computing its calculations. The correlation coefficient is the basic statistic used to determine the relationship between two variables. Factor analysis cannot be run until ‘every possible correlation’ among the variables has been computed (Cattell, 1973).
To understand how factor analysis works, suppose that Xi, Xj …, Xp are variables and F1, F2 …, Fm are factors. For all pairs Xi and Xj, we want to find factors such that when they are extracted, there is an absence of partial correlation between the tests, that is, the partial correlations are zero (Jöreskog & Sörbom, 1979).
Once a correlation matrix is computed, the factor loadings are then analyzed to see which variables load onto which factors. In matrix notation, factor analysis can be described by the equation
R = PCP’ + U2
where R is the matrix of correlation coefficients among observed variables, P is the primary factor pattern or loading matrix (P’ is the transpose), C is the matrix of correlations among common factors, and U2 is the diagonal matrix or unique variances (McDonald, 1985).
From the equation
Rmxm – U2mxm = FmxP F’pxm
Where, Rmxm denotes the correlation matrix, U2mxm is the diagonal matrix of unique variances of each variable, and Fmxp represents the common factor loadings. The left-hand side of the equation represents the correlation matrix of the common parts. Since U2 is the unique variances, when we subtract this out of R then it gives us the common variance (Rummel, 1970). Finding Fmxp can be solved by determining the eigenvalues and eigenvectors of the matrix.
According to Cattell, 1973 the factors are represented by the axes and the variables are lines or vectors illustrated in graphical manner see figure 1. If a factor and a certain variable are in close proximity, this would mean they are associated to each other.
Figure 1. A geometrical representation of factor analysis in two-dimensional space where the blue triangles load onto factor 1 and the green triangles load onto factor 2.
Components of factor analysis
The ‘common factor model’ is a theoretical model on which factor analysis is built. This model postulates that observed measures are affected by underlying common factors and unique factors, and the correlation patterns need to be determined.
Factors are rotated for better interpretation since un-rotated factors are confusing. The aim of rotation is to obtain a final simple structure which attempts to have each variable load on as few factors as possible, but maximizes the number of high loadings on each variable (Rummel, 1970).
There are two types of rotations, orthogonal rotation and oblique rotation. In orthogonal rotation the factors are rotated at 90˚ and are said to be uncorrelated (DeCoster, 1998; Rummel, 1970). It is further sub-divided into Quartimax and Varimax rotation. In quartimax rotation there is the minimization of the number of factors needed to explain each variable
(Gorsuch, 1983). Whereas varimax minimizes the number of variables that have high loadings on each factor and works to make small loadings even smaller.
Oblique rotation is when the factors are different from 90° rotation from each other, and the factors are considered to be correlated. Oblique rotation is more complex than orthogonal rotation, given that it can involve one of two coordinate systems: a system of primary axes or a system of reference axes (Rummel, 1970). Moreover, oblique rotation produces a pattern matrix that comprises the factor or item loadings and factor correlation matrix that includes the correlations between the factors. Oblique rotation techniques can further be divided into direct Oblimin and Promax.
When interpreting the factors, you need to look at the loadings to determine the strength of the relationships. A factors can be clearly demarked by high loadings, but it is also important to inspect the zero and low loadings in order to confirm the identification of the factors (Gorsuch, 1983). For example if you have a factor called ‘anxiety’ and variables that load high on this factor are ‘heartbeat’ and ‘perspiration’, it is also good that extreme variable such as ‘lethargy’ does not load onto this factor. Another option is to choose a significant loading cut-off to make interpretation easier. The signs of the loadings show the direction of the correlation and do not play on the interpretation of the magnitude of the factor loading or the number of factors to retain (Kline, 1994).
September 23, 2019