Skip to content

feat: add a POST /api/report-content Netlify Function to allow users to flag abusive or inappropriate posts β€” the platform currently has no mechanism to report harmful user-generated content, leaving moderators blind to violationsΒ #186

Description

@divyanshim27

πŸš€ Problem Statement

SafeVoice allows anonymous content publishing, which is a strength for protecting user privacy. However, this anonymity also creates a risk: abusive, harassing, or false content can be posted with no way for the community to flag it.

Currently, there is:

  • No "Report" button on story cards in Stories.tsx
  • No backend function to record reports
  • No admin visibility into flagged content
  • No way for the AdminDashboard.tsx to surface reports for review

Without a reporting mechanism, SafeVoice cannot fulfill its mission of being a safe platform β€” toxic content can persist indefinitely without the admin knowing it exists.

βœ… Proposed Solution

1. Create netlify/functions/report-content.cjs

// netlify/functions/report-content.cjs
const { initializeApp, cert } = require('firebase-admin/app');
const { getFirestore } = require('firebase-admin/firestore');

exports.handler = async (event) => {
  if (event.httpMethod !== 'POST') {
    return { statusCode: 405, body: 'Method Not Allowed' };
  }

  const { storyId, reason, reportedBy } = JSON.parse(event.body);

  const validReasons = [
    'harassment', 'hate_speech', 'misinformation',
    'spam', 'self_harm_content', 'other'
  ];

  if (!storyId || !validReasons.includes(reason)) {
    return {
      statusCode: 400,
      body: JSON.stringify({ error: 'Invalid report data' })
    };
  }

  const db = getFirestore();

  // Prevent duplicate reports from same session
  const existingReport = await db.collection('reports')
    .where('storyId', '==', storyId)
    .where('reportedBy', '==', reportedBy)
    .get();

  if (!existingReport.empty) {
    return {
      statusCode: 409,
      body: JSON.stringify({ error: 'You have already reported this content' })
    };
  }

  await db.collection('reports').add({
    storyId,
    reason,
    reportedBy: reportedBy || 'anonymous',
    timestamp: new Date().toISOString(),
    status: 'pending', // pending | reviewed | dismissed | removed
    reviewedBy: null,
  });

  // Auto-flag story if report count exceeds threshold
  const reportCount = await db.collection('reports')
    .where('storyId', '==', storyId)
    .where('status', '==', 'pending')
    .get();

  if (reportCount.size >= 3) {
    await db.collection('stories').doc(storyId).update({
      flagged: true,
      flaggedAt: new Date().toISOString()
    });
  }

  return {
    statusCode: 200,
    body: JSON.stringify({ success: true, message: 'Report submitted for review' })
  };
};

2. Add Report button to story cards in Stories.tsx

// src/pages/Stories.tsx
const [reportModalOpen, setReportModalOpen] = useState(false);
const [selectedStoryId, setSelectedStoryId] = useState('');

const handleReport = async (reason: string) => {
  await fetch('/.netlify/functions/report-content', {
    method: 'POST',
    body: JSON.stringify({
      storyId: selectedStoryId,
      reason,
      reportedBy: sessionId // anonymous session ID
    })
  });
  toast.success('Report submitted. Thank you for keeping SafeVoice safe.');
};

// In JSX, add to each story card:
<button
  onClick={() => { setSelectedStoryId(story.id); setReportModalOpen(true); }}
  className="text-gray-400 hover:text-red-500 transition-colors"
  aria-label="Report this content"
>
  🚩 Report
</button>

3. Surface reports in AdminDashboard.tsx

// src/pages/AdminDashboard.tsx β€” add Reports tab
const [reports, setReports] = useState([]);

useEffect(() => {
  const q = query(
    collection(db, 'reports'),
    where('status', '==', 'pending'),
    orderBy('timestamp', 'desc')
  );
  const unsub = onSnapshot(q, snap => setReports(snap.docs.map(d => ({ id: d.id, ...d.data() }))));
  return unsub;
}, []);

πŸ“ Files to Create / Modify

File Change
netlify/functions/report-content.cjs Create reporting backend function
src/pages/Stories.tsx Add Report button and modal
src/pages/AdminDashboard.tsx Add Reports tab with review workflow
firestore.rules Add rules for reports collection

Suggested labels: enhancement, safety, backend, frontend, level: intermediate

I would like to work on this. Could you please assign it to me?

Metadata

Metadata

Assignees

No one assigned

    Labels

    gssoc'26Contribution for Girlscript Summer of Code'26

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions