Diversified Stock Portfolio Using Clustering Analysis
S&P 500 portfolio construction using K-means clustering on risk/return features.
Overview
This project constructs diversified stock portfolios from the S&P 500 using unsupervised learning. Historical price data is used to compute risk and return features for each stock; K-means clustering groups stocks with similar behavior. Portfolio construction then selects top stocks by Sharpe ratio from each cluster to achieve diversification across clusters and validate performance via backtesting against the S&P 500 index.
Data & Features
Data sources: S&P 500 constituent list and historical stock prices (US equity data fetched via the pipeline). The first 70% of the history is used for model building; the remaining 30% is reserved for validation.
Features used for clustering (all derived from historical data):
Clustering Features
Correlation with S&P 500 index (price correlation).
Beta: sensitivity of stock returns to index returns.
Annualized return (from daily returns, 252-day year).
Annualized volatility (standard deviation of daily returns, annualized).
Sharpe ratio (annualized return / annualized volatility).
Daily change in price (open-to-close) and daily variation (high-to-low), annualized.
Features are scaled (z-score) before clustering. The optimal number of clusters is chosen using the within-cluster sum of squares (elbow method); we use K = 4.
Clustering
K-means is run on the normalized feature matrix with multiple random starts. Cluster membership ensures the portfolio spans different behavior groups rather than concentrating in one segment of the risk/return space.
Portfolio Construction
Two portfolio variants are built:
(1) Diversified by cluster: Within each cluster, stocks are ranked by Sharpe ratio; the top 5 from each of the 4 clusters form a 20-stock portfolio (equal weight per stock).
(2) Top-20 by Sharpe: The top 20 stocks by Sharpe ratio across the full universe, without cluster constraints.
Both are equal-weighted. The diversified-by-cluster portfolio reduces concentration in a single risk/return profile.
Validation & Backtesting
Validation uses the holdout period (last 30% of the data). Daily returns are computed for the portfolio (equal-weighted average of constituent returns) and for the S&P 500 index.
Cumulative returns are plotted for the cluster-based portfolio, the top-20 Sharpe portfolio, and the S&P 500 to compare risk-adjusted performance.
Results
Summary
Cluster statistics
| Cluster | Count | Mean return | Mean vol | Mean Sharpe | Mean beta |
|---|---|---|---|---|---|
| 1 | 46 | -1.67% | 29.60% | -0.045 | 0.749 |
| 2 | 46 | 28.84% | 31.20% | 0.946 | 1.040 |
| 3 | 21 | -6.51% | 52.69% | -0.121 | 1.684 |
| 4 | 86 | 8.88% | 25.38% | 0.375 | 0.795 |
Elbow plot: within-cluster sum of squares
Used to choose the number of clusters K; the “elbow” suggests a good K (here we use K=4).
Return vs volatility by cluster
Each point is a stock (train-period annualized return vs annualized volatility), colored by cluster.
Cluster-wise metrics
Mean annualized return, volatility, and Sharpe ratio by cluster.
Mean return
Mean volatility
Mean Sharpe ratio
Feature correlation matrix
Correlation between clustering features (used for K-means). Helps check redundancy.
| ann_return | ann_vol | ann_sharpe… | ann_daily_… | ann_daily_… | beta | cor | |
|---|---|---|---|---|---|---|---|
| ann_return | 1.00 | -0.17 | 0.93 | -0.67 | -0.15 | -0.12 | 0.61 |
| ann_vol | -0.17 | 1.00 | -0.31 | 0.32 | 0.97 | 0.82 | -0.12 |
| ann_sharpe… | 0.93 | -0.31 | 1.00 | -0.65 | -0.27 | -0.22 | 0.64 |
| ann_daily_… | -0.67 | 0.32 | -0.65 | 1.00 | 0.35 | 0.16 | -0.56 |
| ann_daily_… | -0.15 | 0.97 | -0.27 | 0.35 | 1.00 | 0.79 | -0.12 |
| beta | -0.12 | 0.82 | -0.22 | 0.16 | 0.79 | 1.00 | 0.17 |
| cor | 0.61 | -0.12 | 0.64 | -0.56 | -0.12 | 0.17 | 1.00 |
Portfolios
By cluster: top 5 by Sharpe in each of 4 clusters (equal weight).
Top 20: top 20 stocks by Sharpe ratio overall (equal weight).
Portfolio by cluster (symbols)
DVN, CF, CTRA, CPT, EVRG, CEG, ACGL, COST, AJG, ANET, EQT, AMD, APA, CRWD, DDOG, BRK-B, CB, KO, ED, ATO
Top 20 by Sharpe (symbols)
CEG, ACGL, COST, AJG, ANET, CTAS, FICO, AFL, COR, AVGO, CBOE, ABBV, BLDR, BRO, AZO, ETN, BRK-B, BSX, ERIE, CB
Validation: cumulative returns
Equal-weight portfolio vs S&P 500 index over the validation (holdout) period.