corrupted images

[mindaugas-sasnauskas]

hi, I am getting corrupted/chopped images. what might be causing this, I am trying to build home surveillance system on raspberry pi with some cheap chinese ip cameras. video on vlc doesn't seem to be flaky or anything. I tried websocket example and tried to save images do disk, most of the time I get corrupted or chopped images. so what might be the problem?

[harithFED]

are you manage to solve it?

[mindaugas-sasnauskas]

Nope

[harithFED]

Bro. U try adjust the quality. It reduces the probability to get corrupted
images
On 28 Sep 2016 01:49, "Mindaugas Sasnauskas" [email protected]
wrote:

Nope

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#9 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ARqap7yhqYXzhwPM6Y6eCAg9_M4mOXX7ks5quVc7gaJpZM4I3NOA
.

[serafimdasaudade]

hi,
I got the same problem, and reduce the quality don't have any effect.
Any suggestions?

[harithFED]

FFMpeg.prototype._args = function() { return this.arguments.concat([ '-loglevel', 'quiet' , '-i', this.input , '-r', this.rate.toString() , '-vf', 'fps=25' , '-q:v', this.quality.toString() , '-b:v', '64k' , '-f', 'image2' , '-updatefirst', '1' , '-' ], this.resolution ? ['-s', this.resolution] : []); };

i adjust this thing on rtsp-ffmpeg.js. hope it helps

[Seikon]

Try to use canvas instead of image object. img tag has problem updating the src element in a real time updating enviroment and with large resolution images.

[Seikon]

I will upload an example with canvas Image drawing that works for me with large resolution images and base64 encoding

[Seikon]

Hi all! I have recently uploaded and example with canvas instead of img tag that resolve the problem. You can check in server-canvas.js and index-canvas.html inside of example folder.

[plandevida]

HI,
I tried this using server-canvas.js and I had a corrupted partial image on Chrome and dynamic clipped image in firefox.
Attached screen recording on firefox
screensaved-canvas.zip
.

[harithFED]

my current solution right now is reduce your ip cam video size. as lowest as possible but you can try on which settings are the best.

[Seikon]

Hi daniel, can you put here the code of the client side? Be sure that you are not making any conversion or codification of the image data, only take the array from the server and cast to uint8 array and convert it to blob type. You can see the javascript console output and the network messages from the navigator and check if something is going wrong

[plandevida]

Hi,
I just using the server and server-canvas examples including in repository with the index.html and index-canvas.html code on client side. I'm testing the capabilities of this project before to integrate it into my project.

[Seikon]

So try in the drawImage method use the parameters width and height on index-canvas file. Check also the value of array data and or if the console shows any message of error. I tested this example with two ip cams with big video resolution and works well, so i think it is not a problem of resolution

[harithFED]

@Seikon noted thanks

[Seikon]

@harithFED you are welcome. If you noticed any problem please tell us. For example yesterday I discovered a problem with safari for mobile version and I am working on solve it.

@plandevida Could you solve the problem?

[shyser]

Hi, I also have the problem.
I made a test that I read a local picture instead of ffmpeg stream to send to the browser , then, it works. So, it seems that the point is on server side.

[anselanza]

I'm also getting something similar, even when I write frames directly to disk (.jpg images) rather than streaming via websocket.

Most images look like this (the rest are just "empty" 8KB files):

[Seikon]

Hi! Could you describe in depth how are you using the library? I think both of you have the corrupted images on client side. Manage this problem with canvas example and server canvas instead of targeting the image directly to img HTML tag. I am using the library with large resolution images and i don't have any problem in Chrome, Opera, Firefox, Edge, IE, and in android apps.

Show how you are using the library, also try with png images instead of jpg

[anselanza]

I for one am NOT getting the corrupted images on the client side. Like I said, when I saw the issue (using the canvas demo example) then I tried to just write the frames straight to files using fs.WriteFile and I saw corrupted/partial images.

I get perfect images (in a sequence) if I run:
ffmpeg -i rtsp://username:[email protected]:554/videoMain -f image2 -r 1 -updatefirst 1 test%04d.jpg

[Seikon]

Ok, if you are getting perfect images modifying the ffmpeg parameters maybe you need ajust these parameters by the library directly in code.

BUT! I don't know that you could transform directly bytes array to image file. This is because image are not only a matrix representation of pixels, also contain information about the codification of color. For example, a value of a red pixel in RGB [255,0,0] is not the same that in gray scale [255]. This raw conversion produce chopped and decolorated images because color encoding and other reasons.

Try to transform bytes array to and intermediate type like blob and then write it to disk.

Node compile on C++ so maybe you have the primitive UINT8array type something like this before write to disk:

Example:

var buffer = Buffer.from(yourImageData);
var arraybuffer = Uint8Array.from(buffer).buffer;
fs.write(arraybuffer)

[Seikon]

Here you have a method you write images directly from bytes array.

function saveImage(filename, data){
var myBuffer = new Buffer(data.length);
for (var i = 0; i < data.length; i++) {
myBuffer[i] = data[i];
}
fs.writeFile(ARTWORK_PATH+filename, myBuffer, function(err) {
if(err) {
console.log(err);
} else {
console.log("The file was saved!");
}
});
}
saveImage("image.jpg", yourImageData);

[anselanza]

Still getting the exact same behaviour, even after using your code.

Here's the actual code I'm running for now:

const app = require('express')();
const rtsp = require('rtsp-ffmpeg');
const fs = require('fs');
const path = require('path');

const HTTP_SERVER_PORT = 3001;

let frames = 0;
let uri = 'rtsp://xxx:[email protected]:554/videoMain';
let stream = new rtsp.FFMpeg(
	{
		input: uri,
		rate: 1
	}
);

function saveImage(filename, data){
	var myBuffer = new Buffer(data.length);
	for (var i = 0; i < data.length; i++) {
		myBuffer[i] = data[i];
	}
	fs.writeFile(path.resolve('frames', filename), myBuffer, function(err) {
		if(err) {
			console.error(err);
		} else {
			console.log("The file was saved!");
		}
	});
}

stream.on('data', (data) => {
	frames++;
	console.log(`got ${data.length} data @ ${Date.now()}`);
	let fileName = frames+'.png';
	// console.log(data);
	// fs.writeFile(path.resolve('frames', fileName), data);
	saveImage(fileName, data);
});

app.listen(HTTP_SERVER_PORT, () => {
  console.log('Server listening on port', HTTP_SERVER_PORT);
});

And here's a snippet of the console output:

got 8192 data @ 1493372958929
The file was saved!
The file was saved!
got 8192 data @ 1493372958929
got 8192 data @ 1493372958929
The file was saved!
The file was saved!
got 8192 data @ 1493372958930
The file was saved!
The file was saved!
got 1438 data @ 1493372958930
The file was saved!
The file was saved!
The file was saved!
The file was saved!
got 8192 data @ 1493372959838
got 8192 data @ 1493372959838
got 8192 data @ 1493372959838
got 8192 data @ 1493372959839
got 8192 data @ 1493372959839
got 8192 data @ 1493372959839
got 8192 data @ 1493372959840
The file was saved!
got 8192 data @ 1493372959841
got 8192 data @ 1493372959841
The file was saved!
The file was saved!
The file was saved!
got 8192 data @ 1493372959841
The file was saved!
The file was saved!
got 8192 data @ 1493372959842
The file was saved!
The file was saved!
got 8192 data @ 1493372959842
The file was saved!
The file was saved!
got 8192 data @ 1493372959843
The file was saved!
The file was saved!
The file was saved!
got 8192 data @ 1493372959855
got 2968 data @ 1493372959856
The file was saved!
The file was saved!

And here's what the files typically look like:

[Seikon]

It seems in your code writes the image while you are getting images from the library. Maybe this causes that the function can not write the same data because write in disk is a "slow" proccess and the socket is modifiying the data object at the same time you are writting, that produces the corrupted images.

Before write in disk create a copy of the data object.

Do not use data = dataCopy because it only copy the reference in memory.
Use the method copy of javacript object instead.

[anselanza]

Good advice, thanks! Let me give that a try.

[anselanza]

Nope, still doing the same thing. You can see my complete code below. I'm actually creating a new Buffer and copying it before passing it to the saveFile function.

How do I know when the library finished sending a complete frame?

const app = require('express')();
const rtsp = require('rtsp-ffmpeg');
const fs = require('fs');
const path = require('path');

const HTTP_SERVER_PORT = 3001;

let frames = 0;
let uri = 'rtsp://xxxx:[email protected]:554/videoMain';
let stream = new rtsp.FFMpeg(
	{
		input: uri,
		rate: 1
	}
);

function saveImage(filename, data){

	fs.writeFile(path.resolve('frames', filename), data, function(err) {
		if(err) {
			console.error(err);
		} else {
			console.log("The file was saved!");
		}
	});
}

stream.on('data', (data) => {
	frames++;
	console.log(`got ${data.length} data @ ${Date.now()}`);
	let fileName = frames+'.png';
	// console.log(data);
	// fs.writeFile(path.resolve('frames', fileName), data);
	saveImage(fileName, Buffer.from(data));
});

app.listen(HTTP_SERVER_PORT, () => {
  console.log('Server listening on port', HTTP_SERVER_PORT);
});

[Seikon]

Ok, let me in this days test the code and maybe we can find the problem.

The library receive a complete frame when on data event is fired, i think the data object should be a lock object because of the concurrency to write/read on it, but i am not sure if it occurs in a async context.

Let met 3 days to check it and i will see what is happening

[Seikon]

Sorry, this weekend I was sick and i couldn't try the code. In this week I'll test it and I'll see you if i have the same results on images.

[Seikon]

Hello again @anselanza i test your code and i worked perfectly. Maybe is an issue with your cam, and you only need to adjust the parameters in your code.

What version of ffmpeg are you using?
What is the model of your cam?
What is the codecof your cam?
Your cam is builded based on onvif protocol?

Here You can see my cam configuration:

power frequency
profile
Encode Type
Stream
ChannelName
IP Camera
Copy DeviceName
Resolution

power frequency: 50hz
profile: high
Encode Type: audio&video
Stream: main stream
Resolution: 960x576
BitRateControlType: Variable
BPS: 2560
FPS: 10

[Zombitch]

Got exactly same problem right here. I don't think it depends on camera, because if I edit the rtsp-ffmpeg.js file and overwrite the _args function with this :

FFMpeg.prototype._args = function() {
	return this.arguments.concat([
		'-loglevel', 'quiet'
		, '-i', this.input
		, '-r', this.rate.toString()
		//, '-vf', 'fps=25'
		, '-q:v', this.quality.toString()
		//, '-b:v', '32k'
		, '-f', 'image2'
		, '-updatefirst', '1'
		, '/home/zombitch/streaming.jpg' // MODIFICATION IS RIGHT HERE
	], this.resolution ? ['-s', this.resolution] : []);
};

It works. I mean, I have the whole jpg image in my directory.

I know it doesn't resolve anything but I continue the investigation.

[Zombitch]

Although my previous answer could work. The fix (in my case) was to disabled the quality option (seems not supported and making the image croppped). While disabling the quality option it works perfectly.

You can test by yourself getting my fork : https://github.com/Zombitch/rtsp-ffmpeg

The only thing you'll have to do is to set the quality option to false :

var stream = new rtsp.FFMpeg({input: uri, resolution: '1280x720', quality:false, rate:10});

I'll request a pull on that.

NB : FYI , I got a Hikam A7 camera.

[Seikon]

Hi @anselanza @plandevida @mindaugas-sasnauskas I merged the Zombitch branch to the master branch, so if you get the source code again you can test the solution for cropped images.

[Zombitch]

Thanks @Seikon.

[anselanza]

Thanks, I will give it a try!

[kevinGodell]

Is the data being piped from ffmpeg's stdout? I looked through the code and it seems that when there is a data event, there is an assumption that it is a complete jpeg file. The problem with pipes is that there is no guarantee that the chunk of data is either complete, or arriving in several chunks and has to be reassembled.

Lowering the quality and reducing the width x height can hide the issue if it brings the jpeg file size down to be less than the capacity of the pipe. Also, different machines and OS's have different pipe sizes. If this is the issue, the solution would be using a pipe reader that reads the chunks and looks for the beginning and ending markers for jpeg files.

[Seikon]

That is a very posible solution. Also I think you can change the chunk size from the admin panel of each cam if the provider has implemented it or maybe there is a way to change the chunk limit size from ffmpeg. I will take a look to pipe readers and how to control start and end characters of jpeg and png files.

Thank you.