Machine-learning-project_Comment-Category-Prediction-

Comment Category Prediction Challenge

Overview

This project focuses on building a Machine Learning model to classify comments into different categories using Natural Language Processing (NLP) techniques and feature engineering.

The notebook performs:

• Data preprocessing
• Exploratory Data Analysis (EDA)
• Feature engineering
• Text vectorization using TF-IDF
• Model training and evaluation
• Prediction generation

The project combines both textual features and structured metadata features to improve prediction performance.

---

Features

Data Preprocessing

• Handling missing values
• Cleaning and preparing text data
• Encoding categorical variables
• Date-time feature extraction

Exploratory Data Analysis (EDA)

• Target distribution visualization
• Word count distribution
• Correlation heatmaps
• Feature vs target analysis

Feature Engineering

Text Features

• Word count
• Character count
• Punctuation count
• TF-IDF word features
• Character-level TF-IDF

Metadata Features

• Race
• Religion
• Gender
• Disability
• Upvotes / Downvotes
• Date & Time based features

---

Technologies Used

• Python
• Pandas
• NumPy
• Matplotlib
• Seaborn
• Scikit-learn
• TF-IDF Vectorization
• Jupyter Notebook

---

Machine Learning Workflow

1. Load datasets
2. Perform EDA
3. Handle missing values
4. Extract date features
5. Create text-based features
6. Encode categorical features
7. Apply TF-IDF vectorization
8. Train machine learning model
9. Evaluate model performance
10. Generate predictions

---

Dataset

The project uses:

• train.csv
• test.csv
• Sample.csv

The dataset contains:

• User comments
• Categorical demographic features
• Voting-related features
• Labels for classification

---

Visualizations

The notebook includes multiple visualizations such as:

• Target Distribution Plot
• Word Count Histogram
• Boxplots
• Correlation Heatmaps

These visualizations help understand:

• Class imbalance
• Feature importance
• Relationships between variables
• Text behavior patterns

---

Project Structure

.
├── notebook.ipynb
├── train.csv
├── test.csv
├── Sample.csv
└── README.md

---

Installation

Clone Repository

git clone <your-github-repo-link>
cd <repo-name>

Install Dependencies

pip install pandas numpy matplotlib seaborn scikit-learn

---

Run The Project

Open Jupyter Notebook:

jupyter notebook

Then run the notebook step-by-step.

---

Sample Techniques Used

TF-IDF Vectorization

TF-IDF converts text comments into numerical vectors while preserving word importance.

Feature Engineering

Additional numerical features are extracted from comments and timestamps to improve model learning.

Label Encoding

Categorical columns are transformed into numerical format for machine learning compatibility.

---

Learning Outcomes

Through this project, I learned:

• Natural Language Processing basics
• Text preprocessing techniques
• Feature engineering strategies
• Data visualization
• Machine learning workflow
• Model evaluation techniques

---

Future Improvements

• Apply deep learning models
• Use transformer-based NLP models
• Improve text cleaning pipeline
• Hyperparameter tuning
• Deploy model using Flask or FastAPI

---

Author

Aditya Kumar

If you found this project useful, feel free to star the repository.