Diversified Stock Portfolio Using Clustering Analysis
Overview
This project constructs diversified stock portfolios from the S&P 500 using unsupervised learning. Historical price data is used to compute risk and return features for each stock; K-means clustering groups stocks with similar behavior. Portfolio construction then selects top stocks by Sharpe ratio from each cluster to achieve diversification across clusters and validate performance via backtesting against the S&P 500 index.
Data & Features
Data sources: S&P 500 constituent list and historical stock prices (US equity data fetched via the pipeline). The first 70% of the history is used for model building; the remaining 30% is reserved for validation.
Features used for clustering (all derived from historical data):
Clustering Features
Correlation with S&P 500 index (price correlation).
Beta: sensitivity of stock returns to index returns.
Annualized return (from daily returns, 252-day year).
Annualized volatility (standard deviation of daily returns, annualized).
Sharpe ratio (annualized return / annualized volatility).
Daily change in price (open-to-close) and daily variation (high-to-low), annualized.
Features are scaled (z-score) before clustering. The optimal number of clusters is chosen using the within-cluster sum of squares (elbow method); we use K = 4.
What this shows: Within-cluster sum of squares (WSS) by candidate cluster count K.
How to read it: Look for the elbow where WSS improvement starts flattening; that point suggests a practical K before diminishing returns.
Return vs volatility (K = 4). Color is cluster; hover for ticker.
Feature correlation matrix
| ann_return | ann_vol | ann_sharpe… | ann_daily_… | ann_daily_… | beta | cor | |
|---|---|---|---|---|---|---|---|
| ann_return | 1.00 | -0.10 | 0.95 | -0.69 | -0.10 | -0.03 | 0.66 |
| ann_vol | -0.10 | 1.00 | -0.21 | 0.28 | 0.97 | 0.83 | -0.17 |
| ann_sharpe… | 0.95 | -0.21 | 1.00 | -0.68 | -0.20 | -0.10 | 0.69 |
| ann_daily_… | -0.69 | 0.28 | -0.68 | 1.00 | 0.32 | 0.11 | -0.61 |
| ann_daily_… | -0.10 | 0.97 | -0.20 | 0.32 | 1.00 | 0.79 | -0.18 |
| beta | -0.03 | 0.83 | -0.10 | 0.11 | 0.79 | 1.00 | 0.13 |
| cor | 0.66 | -0.17 | 0.69 | -0.61 | -0.18 | 0.13 | 1.00 |
Clustering
K-means is run on the normalized feature matrix with multiple random starts. Cluster membership ensures the portfolio spans different behavior groups rather than concentrating in one segment of the risk/return space.
| Cluster | Count | Return | Vol | Sharpe | Beta |
|---|---|---|---|---|---|
| 1 | 95 | 10.0% | 24.5% | 0.42 | 0.72 |
| 2 | 37 | 24.9% | 36.8% | 0.68 | 1.26 |
| 3 | 12 | -9.1% | 59.0% | -0.16 | 1.94 |
| 4 | 53 | -7.0% | 31.5% | -0.22 | 0.84 |
Mean return
Mean volatility
Mean Sharpe
Portfolio Construction
Two portfolio variants are built:
(1) Diversified by cluster: Within each cluster, stocks are ranked by Sharpe ratio; the top 5 from each of the 4 clusters form a 20-stock portfolio (equal weight per stock).
(2) Top-20 by Sharpe: The top 20 stocks by Sharpe ratio across the full universe, without cluster constraints.
Both are equal-weighted. The diversified-by-cluster portfolio reduces concentration in a single risk/return profile.
By cluster: top 5 by Sharpe in each of 4 clusters (equal weight). Top 20: top 20 stocks by Sharpe overall.
Portfolio by cluster
COR, ACGL, AJG, CAH, AFL, CEG, AVGO, ANET, FICO, COST, CRWD, AMD, DDOG, COIN, CCL, CTRA, DVN, CHRW, CFG, BG
Top 20 by Sharpe
CEG, AVGO, ANET, FICO, COST, COR, ACGL, AJG, CAH, AFL, ETN, BSX, AZO, APO, DECK, AXON, APH, CTAS, ERIE, BLDR
Validation & Backtesting
Validation uses the holdout period (last 30% of the data). Daily returns are computed for the portfolio (equal-weighted average of constituent returns) and for the S&P 500 index.
Cumulative returns are plotted for the cluster-based portfolio, the top-20 Sharpe portfolio, and the S&P 500 to compare risk-adjusted performance.
Out-of-sample wealth paths indexed to 1 at validation start. Period: 2024-12-24 → 2026-06-29.
Conclusion
K-means clustering on risk–return features produces diversified portfolios that span distinct behavior groups rather than concentrating in a single segment of the cross-section. The cluster-constrained portfolio often trades off peak Sharpe against lower concentration risk relative to a global top-Sharpe screen.
Holdout validation on the last 30% of the sample provides a simple out-of-sample check, but transaction costs and survivorship in the S&P 500 universe are not modeled here.
