Bounty $$ -- FFMediaToolkit.FFmpegException: 'Failed to open video encoder. Error code: -542398533 : Generic error in an external library'

[kobaz]

Getting this randomly. Perhaps some sort of resource issue? It works fine for a while and then stops working.

FFMediaToolkit.FFmpegException
  HResult=0x80131500
  Message=Failed to open video encoder. Error code: -542398533 : Generic error in an external library
  Source=FFMediaToolkit
  StackTrace:
   at FFMediaToolkit.Encoding.Internal.OutputStreamFactory.CreateVideo(OutputContainer container, VideoEncoderSettings config) in /_/FFMediaToolkit/Encoding/Internal/OutputStreamFactory.cs:line 82
   at FFMediaToolkit.Encoding.Internal.OutputContainer.AddVideoStream(VideoEncoderSettings config) in /_/FFMediaToolkit/Encoding/Internal/OutputContainer.cs:line 85
   at FFMediaToolkit.Encoding.MediaBuilder.WithVideo(VideoEncoderSettings settings) in /_/FFMediaToolkit/Encoding/MediaBuilder.cs:line 80
   at IntellaScreenRecord.IntellaScreenRecording.WriteVideo(String path) in C:\intellaApps\intellaScreenRecord\IntellaScreenRecord.cs:line 156
   at IntellaScreenRecord.IntellaScreenRecording.RecordingStart(String path, ScreenRecordingCompleteCallback recordingCompleteCallback) in C:\intellaApps\intellaScreenRecord\IntellaScreenRecord.cs:line 126
   at QueueLib.AgentLoginDialogForm.button1_Click(Object sender, EventArgs e) in C:\intellaApps\QueueLib\AgentLoginDialogForm.cs:line 178
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
   at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
   at intellaQueue.Program.doMain(String[] args) in C:\intellaApps\intellaQueue\src\intellaQueue\Program.cs:line 71
   at intellaQueue.Program.Main(String[] args) in C:\intellaApps\intellaQueue\src\intellaQueue\Program.cs:line 44

---

Here's how it's being used:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Input;

// Dependency: NuGet Install-Package FFMediaToolkit
using FFMediaToolkit;
using FFMediaToolkit.Graphics;
using FFMediaToolkit.Encoding;

using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Runtime.InteropServices;
using System.Threading;
using System.Threading.Tasks;
using Lib;

namespace IntellaScreenRecord
{
    public enum SystemMetric
    {
        VirtualScreenWidth  = 78, // CXVIRTUALSCREEN 0x0000004E 
        VirtualScreenHeight = 79, // CYVIRTUALSCREEN 0x0000004F 
        SM_CYFULLSCREEN = 17,
        SM_CXFULLSCREEN = 16,
    }

    public class IntellaScreenRecording
    {
        public delegate void ScreenRecordingCompleteCallback(IntellaScreenRecordingResult result);

        private static bool m_ffmpeg_init = false;

        /// ///////////////////////////////////////////////////////////////////
        
        // Callbacks
        private QD.QD_LoggerFunction m_logger = null;
        private ScreenRecordingCompleteCallback m_screenRecordingCompleteCallback;

        private int screenWidth, screenHeight;      // Screen size
        private int frameRate = 20;                 // Frame rate of the video

        private string appPath;
        private MediaOutput m_mediaOutput;
        private bool m_currentlyRecording;
        private IntellaScreenRecordingResult m_RecordingResult;

        /// ///////////////////////////////////////////////////////////////////

        [DllImport("gdi32.dll")]
        static extern int GetDeviceCaps(IntPtr hdc, int nIndex);

        [DllImport("user32.dll")]
        public static extern int GetSystemMetrics(SystemMetric metric);

        public enum DeviceCap
        {
            VERTRES = 10,
            DESKTOPVERTRES = 117,        
        }

        private float GetScalingFactor()
        {
            Graphics g                = Graphics.FromHwnd(IntPtr.Zero);
            IntPtr desktop            = g.GetHdc();
            int LogicalScreenHeight   = GetDeviceCaps(desktop, (int) DeviceCap.VERTRES);
            int PhysicalScreenHeight  = GetDeviceCaps(desktop, (int) DeviceCap.DESKTOPVERTRES);
            float ScreenScalingFactor = (float) PhysicalScreenHeight / (float) LogicalScreenHeight;

            return ScreenScalingFactor; // 1.25 = 125%
        }

        public Bitmap TakeScreenShot()
        {
            var sc = GetScalingFactor();

            Bitmap shot = new Bitmap(screenWidth, screenHeight);
            var graphics = Graphics.FromImage(shot);

            graphics.CopyFromScreen(0, 0, 0, 0, shot.Size);                

            return shot;
        }

        private static ImageCodecInfo GetEncoder(ImageFormat format)
        {
            return ImageCodecInfo.GetImageDecoders().FirstOrDefault(codec => codec.FormatID == format.Guid);
        }

        public IntellaScreenRecording()
        {
            InitVariables();
        }

        void InitVariables()
        {
            screenWidth  = GetSystemMetrics(SystemMetric.VirtualScreenWidth);
            screenHeight = GetSystemMetrics(SystemMetric.VirtualScreenHeight);
            //mainWindow.Title = "Screen size: " + screenWidth.ToString() + " ×" + screenHeight.ToString();
            m_currentlyRecording = false;

            if (!IntellaScreenRecording.m_ffmpeg_init) {
                // Where to find FFMPEG
                appPath                 = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
                FFmpegLoader.FFmpegPath = System.IO.Path.Combine(appPath, "ffmpeg");
            }

           IntellaScreenRecording.m_ffmpeg_init = true;
        }
 
        public bool RecordingStart(string path, ScreenRecordingCompleteCallback recordingCompleteCallback)
        {
            if (m_currentlyRecording) {
                return false;
            }

            m_currentlyRecording = true;

            if (System.IO.Path.GetDirectoryName(path) == "") {
                path = System.IO.Path.Combine(appPath, path);
            }

            WriteVideo(path);

            m_screenRecordingCompleteCallback = recordingCompleteCallback;

            m_RecordingResult                   = new IntellaScreenRecordingResult();
            m_RecordingResult.StartTime         = DateTime.Now;
            m_RecordingResult.RecordingFilePath = path;

            return true;
        }
                
        public bool IsRunning() {
            return m_currentlyRecording;
        }

        public bool RecordingStop()
        {
            if (!m_currentlyRecording) return false;

            m_currentlyRecording = false;

            return true;
        }

        private void WriteVideo(string path)
        {
            var settings           = new VideoEncoderSettings(width: screenWidth, height: screenHeight, framerate: frameRate, codec: VideoCodec.H264);
            settings.EncoderPreset = EncoderPreset.Fast;
            settings.CRF           = 17;

            m_mediaOutput = MediaBuilder.CreateContainer(path).WithVideo(settings).Create();

            DoRecording();
        }

        private void DoRecording()
        {
            Task t = Task.Factory.StartNew(() =>
            {
                long starttime = DateTime.Now.Ticks;
                long oldtime   = DateTime.Now.Ticks;
                long delta     = 10000000 / frameRate;          // its 10000000 ticks in 1 seccond
                long curTime   = 0;
                long frcount   = 0;

                while (m_currentlyRecording) {
                    frcount++;

                    var bitmap     = TakeScreenShot();
                    var rect       = new System.Drawing.Rectangle(System.Drawing.Point.Empty, bitmap.Size);
                    var bitLock    = bitmap.LockBits(rect, ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
                    var bitmapData = ImageData.FromPointer(bitLock.Scan0, ImagePixelFormat.Bgr24, bitmap.Size);

                    while (DateTime.Now.Ticks - oldtime < 500000) {
                        Thread.Sleep(5);
                    }

                    curTime = DateTime.Now.Ticks;
                    TimeSpan ts = new TimeSpan(curTime - starttime);

                    m_mediaOutput.Video.AddFrame(bitmapData,ts); // Encode the frame
                    //m_mediaOutput.Video.AddFrame(bitmapData);
                    bitmap.UnlockBits(bitLock);

                    //while (DateTime.Now.Ticks - oldtime < delta)
                    //    Thread.Sleep(1000);
                    oldtime = DateTime.Now.Ticks;
                }

                m_mediaOutput.Dispose();

                m_RecordingResult.Success = true;
                m_RecordingResult.EndTime = DateTime.Now;

                m_screenRecordingCompleteCallback.Invoke(m_RecordingResult);
            });
        }
        
        // QD.QD_LoggerFunction = (string msg, params string[] msgFormat)
        public void SetLoggerCallback(QD.QD_LoggerFunction loggerFn) {
            m_logger = loggerFn;
        }
    }

    public class IntellaScreenRecordingResult
    {
        public DateTime StartTime;
        public DateTime EndTime;

        public string RecordingFilePath;

        public bool Success;
    }
}

[IsaMorphic]

If it only occurs occasionally my best guess is that because you are using Task.StartNew() in DoRecording(), you are using the MediaContainer instance on a different thread than it was created on; you probably have a race condition.

I recommend, instead of DoRecording() returning void, it should instead return a Task that can then be awaited by the caller. You should also refactor your code so that the MediaContainer is created within that same thread at the start (i.e inside the lambda function passed to Task.StartNew()).

Also note that conventionally, to cancel the running recording task, you would use a CancellationToken passed as a parameter to the asynchronous method.

See this page for official Microsoft documentation on how to properly use the .NET Task Parallel Library.

Good luck, and feel free to send the bounty reward my way if this was at all helpful. My PayPal is [email protected]

[kobaz]

Howdy @IsaMorphic,

This was not the issue, but also thanks for the suggestions. I'll send you something for your time.

The cause of the problem was having a height component of the screen resolution that was not divisible by 2 (Happens if you're in a remote desktop session with dynamic sizing enabled.. so you could in theory have a resolution height like 1235)

I found the issue with doing some extensive research on how to get the low level debug logs from ffmpeg. I'll post them up here for future reference. This whole 'Generic Error' popping up with absolutely no other information made this a hard problem to find.

[IsaMorphic]

Thanks for updating me, and you're welcome! I thought this could also be an issue but it relied on an assumption on my part about how your application functioned. I appreciate the money-stuffs, too :D