Introduction

Standard sentiment models output positive/negative/neutral labels or scores. These don't capture nuance: mild positive vs strong positive are both "positive." Contrastive learning creates vector representations where distance encodes sentiment magnitude and nuance.

Contrastive Learning Fundamentals

Train model so that similar items (sentences with similar sentiment) have embeddings close together; dissimilar items have distant embeddings. This creates a continuous sentiment space rather than discrete categories.

SimCLR and Sentiment Vectors

Use SimCLR framework: for a sentence, create two augmented versions (paraphrase, synonym substitution), project both through model, maximize similarity (pull together). Create negative pairs (different sentiment), minimize their similarity (push apart).

Result: sentiment vectors where Euclidean distance reflects sentiment difference. Interpolation between vectors creates gradations of sentiment.

Building Training Data for Sentiment Contrasts

Create triplets: anchor (sentence), positive (similar sentiment), negative (different sentiment). For "great earnings" (positive): positive might be "excellent results" (similar), negative might be "disappointing earnings" (opposite). Use existing sentiment datasets + domain knowledge.

Applications: Intensity Estimation

Embed all sentences from earnings call. Cluster by sentiment vector distance. Sentences close together have similar sentiment; those far apart have different sentiment. Magnitude of vector can encode intensity: stronger vectors = stronger sentiment.

Sentiment Composition

Contrastive vectors enable "sentiment arithmetic": sentiment of document = average of sentence vectors. This automatically weights sentences by importance (longer documents with diverse sentiment produce vectors closer to neutral).

Cross-Domain Transfer

Model trained on general sentiment corpus (movie reviews, news) can transfer to finance. Add fine-tuning on financial data: create finance triplets to specialize vectors. Transfer learning + domain adaptation achieves better results than training from scratch with limited finance data.

Empirical Results

Contrastive sentiment vectors on financial news:

  • Correlation between vector magnitude and next-day returns: 0.35 (vs 0.28 for standard sentiment scores)
  • Ability to distinguish "mildly positive" from "very positive": 82% accuracy
  • Sensitivity to industry-specific language: captures industry-dependent sentiment meanings

Application: Sentiment Distribution Forecasting

Instead of point sentiment estimates, estimate sentiment distribution from vectors. Use kernel density estimation on vector sets from past news. Forecast: will sentiment tomorrow be more intense or moderate? Uncertainty = width of distribution.

Implementation

Use pytorch + pytorch-lightning for contrastive learning framework. Pre-train on SimCLR setup, fine-tune on financial triplets. Store sentence embeddings in vector database (Pinecone, Weaviate) for fast similarity search and clustering.