pca scores and loadings python

Yulia Chive

3 min read

·

08-12-2024

10

0

Share

Understanding PCA Scores and Loadings in Python

Principal Component Analysis (PCA) is a powerful dimensionality reduction technique used to simplify complex datasets by identifying the principal components – new, uncorrelated variables that capture the maximum variance in the data. Understanding PCA involves interpreting both the scores and the loadings. This article will guide you through calculating and interpreting PCA scores and loadings using Python, focusing on practical application and clear explanations.

What are PCA Scores and Loadings?

Imagine your data as a cloud of points in a multi-dimensional space. PCA finds the directions (axes) of greatest variance within this cloud. These directions are the principal components.

• Scores: The PCA scores represent the coordinates of your original data points projected onto these new principal component axes. Each score represents how much each data point contributes to a particular principal component. Essentially, they're the transformed data in the reduced-dimensionality space.

• Loadings: The loadings indicate the contribution of each original variable to each principal component. They show the correlation between the original variables and the principal components. They are essentially the weights assigned to each original variable when forming the principal components. High absolute values indicate a strong contribution.

Performing PCA with Python (scikit-learn)

We'll use the scikit-learn library for PCA calculation. Let's illustrate with a simple example:

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt

# Sample Data (replace with your own)
data = {'Var1': [1, 2, 3, 4, 5],
        'Var2': [2, 4, 1, 3, 5],
        'Var3': [3, 1, 5, 2, 4]}
df = pd.DataFrame(data)

# 1. Standardize the data
scaler = StandardScaler()
scaled_data = scaler.fit_transform(df)

# 2. Apply PCA
pca = PCA(n_components=2) # Reduce to 2 principal components
pca_result = pca.fit_transform(scaled_data)

# 3. Get scores and loadings
scores = pd.DataFrame(pca_result, columns=['PC1', 'PC2'])
loadings = pd.DataFrame(pca.components_.T, columns=['PC1', 'PC2'], index=df.columns)

# Print results
print("PCA Scores:\n", scores)
print("\nPCA Loadings:\n", loadings)

This code first standardizes the data (crucial for PCA) and then applies PCA, reducing the dimensionality to two principal components. It then extracts and displays the scores and loadings.

Interpreting the Results

Let's break down how to interpret the output:

• Scores: Examine the scores DataFrame. Each row represents a data point, and the columns (PC1, PC2 in this case) represent the coordinates of that point in the new principal component space. Points with similar scores on a principal component are similar along that dimension. You can visualize these scores using scatter plots to identify clusters or patterns.

• Loadings: The loadings DataFrame is key to understanding the meaning of the principal components. Each row corresponds to an original variable, and the columns represent the loadings on each principal component.

  • High positive loading: Indicates a strong positive correlation between the variable and the principal component. A high positive loading on PC1 means that variable increases as PC1 increases.
  • High negative loading: Indicates a strong negative correlation. A high negative loading on PC1 means that variable decreases as PC1 increases.
  • Loadings near zero: Indicate a weak relationship between the variable and the principal component.

Visualizing PCA Results

Visualizing the scores and loadings helps with interpretation.

# Scatter plot of PCA scores
plt.figure(figsize=(8, 6))
plt.scatter(scores['PC1'], scores['PC2'])
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.title('PCA Scores Plot')
plt.show()

# Biplot (combining scores and loadings) - requires additional libraries like `matplotlib`
# ... (Code for biplot would be more complex and is omitted for brevity.  Search for "biplot python pca" for examples)

A biplot overlays the scores and loadings on a single plot, providing a visual representation of how the original variables contribute to the principal components and how the data points are distributed in the reduced-dimensional space.

Choosing the Number of Components

The number of principal components to retain depends on your data and goals. You can use techniques like the explained variance ratio to determine how many components capture a sufficient amount of the total variance. pca.explained_variance_ratio_ provides this information.

Conclusion

PCA scores and loadings provide valuable insights into your data. By carefully interpreting these outputs and visualizing them, you can reduce dimensionality while retaining crucial information, uncovering hidden patterns, and gaining a deeper understanding of your dataset. Remember to always standardize your data before performing PCA. This ensures that variables with larger scales don't disproportionately influence the results.