Getting Started with Mantissa Log

What is Mantissa Log?

Mantissa Log is an AI-powered security log analysis platform that transforms how you monitor and investigate security events in your cloud infrastructure.

Key Features:

Natural Language Queries: Ask questions about your logs in plain English instead of writing complex SQL queries
Intelligent Detection: AI-powered detection rules that adapt and learn from your environment
Multi-Source Integration: Unified view across CloudTrail, VPC Flow Logs, GuardDuty, and custom application logs
Smart Alerting: Context-aware alerts with automatic enrichment and intelligent routing
Serverless Architecture: Fully serverless deployment on AWS with automatic scaling
Cost Effective: Pay only for what you use with optimized query execution

Who is it for?

Security teams needing faster log analysis and threat detection
DevOps teams monitoring cloud infrastructure
Compliance teams requiring audit trail analysis
Organizations wanting to reduce SIEM costs
Teams looking to leverage AI for security operations

Quick Start

Prerequisites

Before you begin, ensure you have:

AWS account with administrative access
Terraform >= 1.5.0
AWS CLI >= 2.0
Python >= 3.11
Git

See Prerequisites Guide for detailed requirements.

One-Command Deployment

Clone the repository and run the deployment script:

git clone <repository-url>
cd mantissa-log-dev
bash scripts/deploy.sh

The interactive deployment wizard will guide you through configuration:

Environment name (dev/staging/prod) [dev]: dev
AWS Region [us-east-1]: us-east-1
LLM Provider (bedrock/anthropic/openai) [bedrock]: bedrock
Enable VPC Flow Logs ingestion? (y/n) [y]: y
Enable GuardDuty integration? (y/n) [y]: y

Deployment typically takes 5-10 minutes.

Your First Query

After deployment, you can query your logs using natural language:

Using the API:

# Get your API endpoint from deployment outputs
API_ENDPOINT=$(cat terraform-outputs.json | jq -r '.api_endpoint.value')

# Authenticate (using Cognito credentials)
TOKEN="your-jwt-token"

# Ask a question
curl -X POST "$API_ENDPOINT/query" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Show me failed login attempts in the last 24 hours",
    "execute": true
  }'

Example Queries:

"Show me all root account activity this week"
"What are the most common API errors in the last hour?"
"List all S3 bucket deletions from the past month"
"Show me failed authentication attempts by source IP"
"Which IAM users created new access keys recently?"

Your First Detection Rule

Detection rules automatically scan your logs for security issues.

1. Create a rule file:

cat > rules/custom/suspicious-logins.yaml <<EOF
name: "Suspicious Login Pattern"
description: "Multiple failed logins followed by success"
enabled: true
severity: "high"
category: "access"

query: |
  WITH failed_attempts AS (
    SELECT
      useridentity.principalid as user,
      sourceipaddress as ip,
      COUNT(*) as failures
    FROM cloudtrail
    WHERE eventname = 'ConsoleLogin'
      AND errorcode IS NOT NULL
      AND eventtime > CURRENT_TIMESTAMP - INTERVAL '1' HOUR
    GROUP BY useridentity.principalid, sourceipaddress
    HAVING COUNT(*) >= 3
  ),
  successful_logins AS (
    SELECT
      useridentity.principalid as user,
      sourceipaddress as ip
    FROM cloudtrail
    WHERE eventname = 'ConsoleLogin'
      AND errorcode IS NULL
      AND eventtime > CURRENT_TIMESTAMP - INTERVAL '1' HOUR
  )
  SELECT
    f.user,
    f.ip,
    f.failures,
    'Multiple failed login attempts followed by success' as reason
  FROM failed_attempts f
  INNER JOIN successful_logins s
    ON f.user = s.user AND f.ip = s.ip

threshold:
  count: 1
  window: "1h"

metadata:
  mitre_attack:
    - "T1110"  # Brute Force
  tags:
    - "authentication"
    - "brute-force"
EOF

2. Upload the rule:

RULES_BUCKET=$(cat terraform-outputs.json | jq -r '.rules_bucket.value')
aws s3 cp rules/custom/suspicious-logins.yaml s3://$RULES_BUCKET/rules/custom/

3. Rule executes automatically:

The detection engine runs every 5 minutes and will automatically:

Execute your rule against recent logs
Generate alerts if matches are found
Route alerts to configured destinations (Slack, PagerDuty, email)

4. Monitor rule execution:

# Check detection engine logs
aws logs tail /aws/lambda/mantissa-log-detection-engine --follow

Key Concepts

Log Sources

Mantissa Log ingests and analyzes logs from multiple sources:

AWS Native Sources:

CloudTrail: AWS API activity and account events
VPC Flow Logs: Network traffic metadata
GuardDuty: AWS threat detection findings
CloudWatch Logs: Application and service logs

Custom Sources:

Application logs in JSON format
Third-party security tools
Custom event streams

All logs are stored in S3, cataloged in AWS Glue, and queried via Amazon Athena.

How it works:

Logs written to S3 bucket (organized by source type and date)
Glue crawler discovers schema and creates tables
Detection rules query tables using Athena
Natural language queries translate to SQL and execute

See Log Sources Guide for setup details.

Detection Rules

Detection rules are YAML files that define what to look for in your logs:

name: "Rule Name"
description: "What this rule detects"
enabled: true
severity: "critical|high|medium|low|info"
category: "access|network|data|compliance|threat"

query: |
  SELECT ... FROM ... WHERE ...

threshold:
  count: 1          # How many matches trigger an alert
  window: "5m"      # Time window to evaluate

metadata:
  mitre_attack: ["TA0001"]
  tags: ["aws", "security"]

Rule Lifecycle:

Write rule YAML file
Upload to S3 rules bucket
Detection engine loads rules every cycle
Rule queries execute against log tables
Matches generate alerts
Alerts route to destinations

Built-in Rules:

Mantissa Log includes detection rules for:

Root account usage
IAM policy changes
S3 bucket exposure
Security group modifications
Unusual API activity
Failed authentication patterns
Data exfiltration indicators

See Detection Rules Guide for creating custom rules.

Alerts

When detection rules find matches, they generate alerts with:

Alert Structure:

Title: Brief description of the issue
Severity: critical, high, medium, low, info
Description: Detailed explanation
Evidence: Query results that triggered the alert
Context: Enriched data (IP geolocation, related events)
Metadata: Rule information, MITRE ATT&CK mappings

Alert Enrichment:

Alerts are automatically enriched with:

IP geolocation data
Threat intelligence lookups
Related alerts from same source
Historical context

Alert Lifecycle:

Detection rule matches log entries
Alert generated with details
Alert enriched with context
Alert routed to destinations
Alert stored in DynamoDB
Alert can be acknowledged/resolved via API

See Alert Routing Guide for configuration.

Natural Language Queries

Instead of writing SQL, ask questions in plain English:

How it works:

You ask a question: "Show me failed logins"
LLM translates to SQL using your schema
SQL is validated for safety
Query executes in Athena
Results returned in structured format

Session Support:

Queries can be conversational:

You: "Show me CloudTrail events from the last hour"
System: [Returns SQL and results]

You: "Filter to just S3 events"
System: [Refines previous query]

You: "Group by user"
System: [Further refines query]

Safety Features:

SQL validation prevents destructive operations
Only SELECT queries allowed
Table access controls enforced
Query cost limits applied

Supported Questions:

Time-based queries: "last hour", "yesterday", "this week"
Filtering: "only errors", "from IP 1.2.3.4", "by user admin"
Aggregations: "count by user", "top 10", "group by hour"
Joins: "correlate with VPC flow logs"

See LLM Configuration Guide for provider setup.

Next Steps

Now that you have Mantissa Log deployed:

1. Configure Log Sources

Enable additional AWS log sources:

2. Set Up Alerts

Configure where alerts should go:

3. Customize Detection

Review and customize detection rules:

4. Explore Your Data

Start querying your logs:

5. Operational Excellence

Establish operational practices:

Follow the Operations Runbook
Set up Monitoring
Plan for Scaling

Getting Help

Documentation:

Common Issues:

Support:

Check GitHub Issues
Review CloudWatch Logs
Run diagnostic commands from troubleshooting guide

Architecture Overview

Understanding the components:

Data Flow:

Log Sources (CloudTrail, VPC Flow, etc.)
    |
    v
S3 Buckets (Organized by source/date)
    |
    v
Glue Catalog (Schema discovery)
    |
    v
Athena (SQL query engine)
    ^         ^
    |         |
Detection    Natural Language
Engine       Query Handler
    |              |
    v              v
Alerts        Results
    |
    v
Alert Router
    |
    v
Destinations (Slack, PagerDuty, Email)

Components:

Detection Engine Lambda: Executes detection rules every 5 minutes
LLM Query Lambda: Translates natural language to SQL
Alert Router Lambda: Routes alerts to configured destinations
S3 Buckets: Store logs, rules, and query results
DynamoDB: Track detection state and query sessions
Glue Catalog: Metadata for log tables
Athena: Query execution engine
EventBridge: Schedule detection cycles
Cognito: User authentication
API Gateway: REST API endpoints

All components are serverless and scale automatically based on load.

Cost Management

Typical monthly costs for different scales:

Small (< 1M events/month): $40-105 Medium (1M-10M events/month): $105-360 Large (10M+ events/month): $360-1,600

Primary cost drivers:

Athena: $5 per TB scanned
S3 storage: $0.023 per GB
Lambda executions: $0.20 per 1M requests
DynamoDB: $1.25 per million writes

Cost optimization tips:

Use partitioning in Glue tables
Convert logs to Parquet format
Set S3 lifecycle policies
Tune detection frequency
Use Athena query result reuse
Monitor with CloudWatch billing alarms

See Prerequisites Guide for detailed estimates.

Security Considerations

Mantissa Log is designed with security in mind:

Data Protection:

All S3 buckets encrypted at rest (AES-256)
TLS 1.2+ for all API communication
Secrets stored in AWS Secrets Manager with KMS encryption

Access Control:

IAM roles for all service-to-service communication
Cognito for user authentication
API Gateway with JWT validation
Least privilege Lambda execution roles

Compliance:

All data stays in your AWS account
No data sent to external services (except LLM APIs if configured)
CloudTrail audit logs for all access
VPC deployment option available

Best Practices:

Enable MFA for Cognito users
Rotate API keys regularly
Review IAM policies quarterly
Monitor CloudTrail for admin actions
Use VPC endpoints for sensitive environments

Frequently Asked Questions

Q: Can I use my own LLM API key? A: Yes, Mantissa Log supports AWS Bedrock (no API key needed), Anthropic API, and OpenAI API. See LLM Configuration.

Q: How much does this cost to run? A: For typical usage (< 1M events/month), expect $40-105/month. See Cost Estimates.

Q: Can I add my own application logs? A: Yes, write JSON logs to S3, create a Glue table, and write detection rules. See Custom Logs.

Q: How do I create custom detection rules? A: Write YAML files following the rule schema and upload to S3. See Detection Rules Guide.

Q: Is my data secure? A: Yes, all data stays in your AWS account with encryption at rest and in transit. See Security Considerations.

Q: Can I deploy to multiple AWS accounts? A: Yes, deploy independently to each account or use cross-account log aggregation. See Scaling Guide.

Q: How do I update to the latest version? A: Run bash scripts/update.sh to pull updates and redeploy. See Update Guide.

Q: What if I want to delete everything? A: Run bash scripts/destroy.sh for complete cleanup. See Destroy Guide.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting Started with Mantissa Log

What is Mantissa Log?

Quick Start

Prerequisites

One-Command Deployment

Your First Query

Your First Detection Rule

Key Concepts

Log Sources

Detection Rules

Alerts

Natural Language Queries

Next Steps

Getting Help

Architecture Overview

Cost Management

Security Considerations

Frequently Asked Questions

FilesExpand file tree

getting-started.md

Latest commit

History

getting-started.md

File metadata and controls

Getting Started with Mantissa Log

What is Mantissa Log?

Quick Start

Prerequisites

One-Command Deployment

Your First Query

Your First Detection Rule

Key Concepts

Log Sources

Detection Rules

Alerts

Natural Language Queries

Next Steps

Getting Help

Architecture Overview

Cost Management

Security Considerations

Frequently Asked Questions