[BUG]Mutex leak in add_to_queue() upon memory allocation failure

[hanqing2025]

Describe the Bug

A mutex leak in add_to_queue() causes permanent deadlock when memory reallocation fails. The function acquires a mutex but returns without releasing it on error paths, preventing all subsequent queue operations.

Location: obe.c (lines 187-192)

int add_to_queue(obe_queue_t *queue, void *item)
{
    void **tmp;

    pthread_mutex_lock(&queue->mutex);  // Lock acquired
    tmp = realloc(queue->queue, sizeof(*queue->queue) * (queue->size+1));
    if (!tmp)
    {
        syslog(LOG_ERR, "Malloc failed\n");
        return -1;  // BUG: Returns without unlock!
    }
    queue->queue = tmp;
    queue->queue[queue->size++] = item;

    pthread_cond_signal(&queue->in_cv);
    pthread_mutex_unlock(&queue->mutex);  // Never reached on error

    return 0;
}

Impact:

• First thread hitting the realloc() failure holds queue->mutex forever
• All subsequent threads calling queue operations block permanently on pthread_mutex_lock()
• Complete encoder pipeline deadlock → Denial of Service
• Easily triggered under memory pressure conditions

Execution Flow:

Thread 1:
  → pthread_mutex_lock(&queue->mutex)    // Lock acquired
  → tmp = realloc(...)                    // Memory allocation fails (OOM)
  → if (!tmp)                             // Condition TRUE
  → return -1                             // Exit without unlock
  → queue->mutex remains locked forever

Thread 2, 3, ...N:
  → pthread_mutex_lock(&queue->mutex)    // Attempts to acquire lock
  → BLOCKS permanently (waiting for Thread 1 to release)
  → Complete deadlock state

This occurs when the system is under memory pressure and realloc() fails to allocate memory for expanding the queue. The function acquires the mutex, attempts reallocation, and returns early without releasing the lock.

CWE Classification:

• CWE-667: Improper Locking
• CWE-833: Deadlock
• CWE-772: Missing Release of Resource after Effective Lifetime

---

I would appreciate it if you could review and confirm this potential issue. Thank you for your time and for maintaining this project!