Financial Sentiment Analysis Project

This project is a simple implementation of a financial sentiment analysis model using Python. It classifies financial news headlines into one of three categories: positive, negative, or neutral.

This project was built as a hands-on introduction to Natural Language Processing (NLP) for quantitative finance applications.

Features

• Text Preprocessing: Cleans text data by removing stop words, lemmatizing, and removing non-alphabetic characters.
• Feature Extraction: Uses TfidfVectorizer from Scikit-learn to convert text into numerical features.
• Machine Learning Model: Employs a Multinomial Naive Bayes classifier to predict sentiment.
• Evaluation: The model's performance is evaluated using an accuracy score, classification report, and a confusion matrix.

Dataset

This project uses the "Financial PhraseBank" dataset, which can be found on Kaggle. It contains sentences from financial news that have been manually labeled with a sentiment.

• Link: Financial Sentiment Analysis Dataset

Results

The model was trained on 80% of the data and tested on the remaining 20%.

• Accuracy: ~67.6%
• Key Findings:
  • The model is highly effective at identifying neutral statements (94% recall).
  • It performs reasonably well on positive statements (53% recall).
  • The model struggles significantly with identifying negative statements (3% recall), often misclassifying them as neutral.

This indicates that while the model can identify clear positive or neutral language, it has difficulty with the nuance of negative financial news.

How to Run This Project

1. Clone the repository:

   git clone https://github.com/YOUR_USERNAME/YOUR_REPOSITORY_NAME.git
   cd YOUR_REPOSITORY_NAME
   

2. Install the dependencies:

   pip install -r requirements.txt
   

3. Download NLTK data: Run the following in a Python interpreter:

   import nltk
   nltk.download('punkt')
   nltk.download('stopwords')
   nltk.download('wordnet')
   

4. Download the dataset:

   • Download data.csv from the Kaggle link above.
   • Place the data.csv file in the root of the project folder.

5. Run the script:

   python sentiment_analyzer.py
   

Future Improvements

• Use a more advanced model: Implement models like Logistic Regression, SVM, or even deep learning models like an LSTM or a transformer (e.g., FinBERT).
• Address class imbalance: The model's poor performance on the negative class could be due to having fewer negative examples. Techniques like oversampling (e.g., SMOTE) could help.
• Improve text preprocessing: Experiment with different techniques, such as bigrams or trigrams.