Friday, March 7, 2008

Handling corrupt JPEGs

If you haven't it figured out, yet, I'm lazy. But, I've been taking my card out of the PC reader and putting it in my camera to take a picture every time I want to get the card to go onto the network. I decided to just create a JPEG on the card instead of actually taking pictures. The quickest way to do that was just to dd some random gunk into a JPEG like this:
    dd if=/dev/urandom of=/media/EYE-FI/dcim/100dchef/dsc_0000.jpg
That works, technically. The card even tries to upload it. But, the server gets a bit annoyed because, somehow, I guess random data isn't a valid JPEG very often. :)
[00:54] Starting to upload "/public/DCIM/100DCHEF/DSC_0000.JPG"...
[00:54] Server has 0 of 77312 bytes for DSC_0000.JPG.tar (file ID 779166).
[00:55] Uploaded 77312 bytes for DSC_0000.JPG.tar. Waiting for the response...
[00:55] Probing for listener at 10.6.0.60:59278.
[00:56] Returning from upload function with error 28.
[00:56] Starting to upload "/public/DCIM/100DCHEF/DSC_0000.JPG"...
...
It will apparently sit in that loop for quite a while. It did until I stopped it by deleting the picture. I bet the card does not quite have the resources to validate JPEGs itself, so it just uploads the raw files to the server. The server returns with that "error 28" and the persistent little card retries.

It might be nice to have a limit on these retries. But, I don't think it's quite that simple. You want the card to be persistent. Wireless isn't exactly a lossless transmission medium, and its bound to have problems uploading from time to time. It's a matter of figuring out when to retry and how much. It might be reasonable, though, to have some special handling of corrupt files. TCP should keep the files consistent during transmission, and I would imagine that most of the corrupt files that the server sees are also corrupt files on the card. There isn't much reason to keep trying to upload these.

1 comment:

Berend said...

I would imagine that most of the corrupt files that the server sees are also corrupt files on the card.

One would hope and wish so, but it's unfortunately not the case when the TCP/IP layer can't provide the warms and fuzzies of not having bugs that result in end-to-end consistency problems.

Until the "X days since last accident" (days since last lwIP bug) on our construction site (firmware) reaches some large value of X, hard hats (end-to-end consistency checks) will continue to be required.

Berend