uploadComplete not fired on large files

Hi, I'm tryng this great script, and I'm having some problems on large files.

When I upload small files (about 5 MB) everithing works fine, when I upload large files (150 MB) the upload is completed and the file goes where I want, but the uploadComplete function is not fired.

I'm pretty sure that is not a php problem (if was so the file was not uploaded, and after the upload I print the filename).

It's a time problem? Maybe with large files the script doesn't wait...

I'm trying with firefox 2 and flash player 9.

Thank you

gyphie's picture

It is important to specify

It is important to specify which SWFUpload version you are using. The Event names have changed slightly in v2.

According to Adobe's Flash documentation the max file size allowed for uploading is 100MB but I have successfully updated 2GB files without problems.

The only thing I can think of is that the upload script is timing-out for large files. That doesn't quite fit though. But you might look at set_time_limit and give that a try.

On the top of my

On the top of my swfupload.js there is this comment:
* SWFUpload Revision 7.0 by Jacob Roberts, Oct 2007, http://linebyline.blogspot.com

I was looking for the official site, but on google i've found only the oldsite, then I've downloaded it from here: http://swfupload.praxion.co.za/

I think it's not a problem of set_time_limit related cause the upload is right, i can see my large files (over 150Mb) uploaded correctly.

I'll do a try, you can confirm is the right version of swfupload?

Thank you very much, Bye

edit: I've downloaded the 2.0.1 version, I've tried the demo and it works, can you suggest me what type of debug can I do?
There's a way to see the comunication between tha flash object and the php file?

thank you again

re-edit:
I've tried to modify the application demo.
If in the upload.php file I put :

move_uploaded_file($_FILES['Filedata']['tmp_name'], 'd:/test.avi');

(the location of the uploaded file will be in another disk different from the one that stores the php temp dir where the files goes)

The time consumed in this operation is approx 40 seconds, and after this time I find my file in the right location, but it seems that the uploadSuccess function don't wait enough.

Can you try for me?
I've did this try:

file upload.php

<?php
set_time_limit(0);
sleep(60);
echo "UPLOAD COMPLETED";

Then I called it with my browser, and after 60 seconds I can se the echo, no errors regarding timeout.

Then I tried with the application example, calling this upload.php and if I reduce the number of seconds I can see the result of the uploadSuccess function.

Thank you

gyphie's picture

Well, first of all I

Well, first of all I strongly recommend upgraded to v2.0.1 (that's my easy out right Eye-wink

I'll do some testing with long waits and see if I can reproduce the issue you are having.

Whatever you do Revision 7.0 is not a version you want to stay with. It was quickly replaced by r7.2b3 because of several bugs.

Now I've switched to version

Now I've switched to version 2.0.1, the flash8 version seems to fire everytime the function uploadSuccess, but if the time of wait is long I receive "undefined" in server data.

The flash 9 version, doesn't wait for the response and doesn't fire the upload success function.

Have you did my try?

Simply put a sleep(60) on the upload.php of the application demo example..

Thank you very much.

bye

gyphie's picture

Server Data is not supported

Server Data is not supported in Flash 8. I'll check out your sleep(60) suggestion

You got time to try ? I'm

You got time to try ?
I'm going to close the project and I've to solve it.

Sorry, I don't want to be impolite...

I've find a solution. Don't

I've find a solution.

Don't know if it is the cleanest way, but it works.

Take the application demo example.

In upload.php I don't return the result, but a blank page then the uploadSuccess is never fired.
In the upload.php I write a file "upload_completed" after the move_uploaded_file instruction.

On the top of the handler.js file I declare a variable "interval", then in the
uploadProgress function :


if (percent === 100) {
progress.SetStatus("Completing...");
progress.ToggleCancel(false);
progress.ToggleCancel(true, this, fileObj.id);
if(interval == 0) {
interval = setInterval('checkUpload('')', 3000);
}
} else {
....
....

The checkUpload function does an ajax call to a php file that returns true if the "upload_completed" file exists, and in the callback function of the ajax call I do the clearInterval on the checkUpload function and display my message for upload completed.

Hope this can be useful for other users that experience this problem.

Bye

Hi there, I'm having the

Hi there,

I'm having the same problem... although the solution posted here doesn't sound that appealing to me, and my problem seems to be showing up round the 5MB mark!

I do think it could have something to do with the response time... The reason I suspect this is that I am doing a lot of image processing once the file is uploaded, and before I send a response. This can take up to a minute with some files because I'm extracting exif, as well as creating thumbnails for pdfs, psds, tiffs, etc. This could explain why some people are experiencing problems with 150mb files, and i'm experiencing problems with 5mb files. Everything works perfectly fine with smaller files.

fyi:
- I am using rails
- the problem occurs on windows/linux regardless
- tested on on ff, ie, safari. Problem exists on all of them.
- the file is uploaded, processed, and stored correctly, but communication between flash and the server seems to be cut at some point.

Thanks,
Jonzo.

Hi jonzo, I think you're

Hi jonzo, I think you're right, cause I can create the problem if in the server side file I sleep for a long time.

I think the solution I found can fit also for you.

The idea on this method is to create a file on the server when your operation is finished, and on the client side doing an ajax request that check if this file exist.
If you want you can put some information on the file that you can read when the file is found.

I repeat, don't know if is the cleanest way, but... it works Smiling

Bye

gyphie's picture

I did a little bit of

I did a little bit of testing. I found that after about 30 seconds of no activity Flash terminates the connection (at least on its end). It looks like the PHP script finishes and returns its output. It looks like PHP still thinks the connection is open.

It's just one flash bug after another. A quick Google search and review of the ActionScript documentation did not turn up any information.
--

My recommendation is to avoid doing a lot of processing on the file. I would do something like what gianiaz suggested. But I'd probably allow the file upload to finish and return properly and then use AJAX to begin the post-processing stuff (by calling some other script that will do this).

--

Another solution is to return some data every 10 seconds or so. Implementing this depends on your server-side language and server configuration. But, I was able to make the Application Demo survive a "sleep" of 5 minutes using the following:

<?php
set_time_limit(0);

$times_to_loop = 30;
$time_to_sleep = 10;

for ($i=0; $i < $times_to_loop; $i++) {
// The "sleep" represents the server doing work.
// I let the server "work" for 10 seconds and then send a "."
sleep($time_to_sleep);

echo ".";
ob_flush(); // I need to flush to try to make sure the data is not buffered.
flush();
}

echo "I finished working for: " . ($times_to_loop * $time_to_sleep) . " seconds";
?>

The server output will look like: "..............................I finished working for: 300 seconds". In my uploadSuccess handler I have to remove all the "." characters from the beginning of my server_data.

Implementing this in a real application might be difficult.

Is there a solution?

Hi!

I've downloaded and installed it locally, successing in small file uploads, but larger ones (i.e. 80MB) cause exactly this behaviour. I'm using the latest verison (2.0.2). Also, I checked out the multi-instance demo (http://demo.swfupload.org/multiuploaddemo/) where I uploaded the same file (80MB) up to 100% with my small upstream (256kBit => 25kB/sec), but without success information.
Since I want to let users upload files up to the maximum (2GB?), I'm strongly interested in a solution.

Regards,
Harald

How to know who's wrong ? Flash or Server ? No Huge file upload

Hi, I've quite the same problem with the 2.0.2

I've isolated the multi-instance demo script (flash8) with the nice graceful degradation plugin ! (real cool stuff !)

The upload is 100% fine with 7~8Mo files
Up to this size, the script shows "Complete"
(my upload bandwidth stays busy for a couple of seconds after)

Just tested the 2.1 Beta... same thing...
Note : the debug module says (after the upload network activity stopped) that the upload is in progress and cannot upload another file

And finally, there's no file in my folder !!!

How to know if this is a problem of server or plugin ?
Is there a soluce ?
Thanks for help

Solution for Rails?

Hello!
Did anybody find a solution to this problem? I did some tests, but in Rails I cannot write to the output buffer and flush the data to the client.

Thanks for your help,
der flo

gyphie's picture

30 second timeout

I'm not familiar with the Rails particulars. The issue is that you have a 30 second timeout in which to complete all server side processing. There are 2 general solutions:

1) Send data before the 30 second timeout (flushing buffers, etc).
2) Do the long running process in a separate thread and allow the main request to return. The downside is that if there is an error you can't return it.

You may also want to check your server's script execution limits. In PHP the default is 30 seconds and you could be running into an issue there as well.

I took option #2

I ended up deferring processing so that I could return a response straight away. This not only solves the time out issue, but I can start uploading the next file while the other file is processing Smiling It all seems to work well so far.

So, although it takes a little more work, in the end it's a better solution anyway... (that is, if you're uploading multiple files)

@gyphie 1) : I'm not able to

@gyphie 1) : I'm not able to flush buffers, I think this is not intended in Rails.
@gyphie 2), @jonzo : Sure, this is a possible solution but it's quite much work to switch our upload-area to async processing. I will drop this task to my todo-list Eye-wink
@gyphie 30-sec-timeout : IMHO there is no such limit in Rails.

I will try gianiaz's solution, but I'm not sure wheter I understand it.
His "checkUpdate()" function ( "interval = setInterval('checkUpload('')', 3000);" ) does an AJAX call, but via JavaScript, not via the open Flash-connection to the server ... I'll give it a try ...

Thanks for your answers,
der flo

Same problems - solved...

I was experiencing the same IO error on large file uploads using the Application Demo. I modified upload.php to simply move_uploaded_file and return it's location, then passed that to a separate ALAX call to handle the thumbnail creation and display - ALL TO NO AVAIL! The AJAX handler would never succeed. It did expose the actual problem which was not a timeout during upload or in waiting for the file processing, but that PHP was maxing out it's memory limit attempting to create the jpeg using imagecreatefromjpeg(). By increasing the php memory limit, I was able to upload AND process larger file-sizes. Remember that php needs more virtual memory than the file-size itself to load and create the new image.

I'm not an image/graphics programmer, but.. example:
If your jpg is 2000X2000 pixels, that is 4,000,000 pixels.
If stored true color you need (i think...)
1 byte R
1 byte G
1 byte B
1 byte alpha channel (transparancy)

So that is 4,000,000 times 4 bytes is approx. 16 megs.

(Note: I think PHP will attempt to load it all into memory.)

To see if this is the problem with your instance, remove the "@" character from

$img = @imagecreatefromjpeg($_FILES["Filedata"]["tmp_name"]);

so php will return and log the error.

I hope this saves someone from a couple of hours of hunting down this problem.

Ruby on Rails Upload Issue

Hi All,

I have been working on the same problem with SWFUpload (which is awesome!)...

I am uploading some files and running a long processing step immediately after upload. This step is essential to the functionality, so I would prefer to process after each upload, rather than to make it into a second step. I do not mind having users wait a couple minutes to run this long process.

I am running into this 30 second timeout/upload problem. What I have been trying is to send data so the SWFUpload does not close its connection. I am doing this as follows (pseudocode):

def create
extract = Thread.new { doExtract }
hack = Thread.new {
loop do
if extract.alive?
#do something
# ???? #
sleep 5
else
break
end
end
}
end

So basically, I spawn two threads, with the second one checking to see every 5 sec, if the extract thread is still alive. The problem is, I don't know what command to use to send something back to avoid termination. I tried outputting to the console using (and even trying to flush the buffer):
puts "Alive"
$stdout.flush

but to no avail -- what call do I need to make here in order to tell the uploader to not terminate?

Thanks!

Rails

@aviir: Actually it is not "prio 1" on my TODO-list.
But perhaps I've found a (theoretically) solution for our problem. Your missing a streaming function for the output buffer? Here you are: "send_data" ( http://api.rubyonrails.com/classes/ActionController/Streaming.html )
Let me know if it worked!
Nevertheless my opinion is that async processing would be much better than such hacks.

Ciao,
der Flo