UPDATE: I found 2 bugs in upload progress module. If you have already installed. update to at least 0.1 version: http://github.com/drogus/apache-upload-progress-module/commits/0.1
I’ve installed mod passenger on my server recently. It’s really great software. Now I don’t have to worry about monitoring, nginx proxy, load balancing, big file uploads… and it’s fast! With Ruby Enterprise Edition it’s even faster.
Personally I don’t care about people saying that phusion wants to promote themselves on REE as long as it gives faster ruby with lower memory use (but yes, I know, REE is not best choice for a name :).
After installing I’ve realised that my shiny upload progress bar (thanks to Upload Progress Module for nginx) was not working. Oh gods! What a tragedy!
But fear not. I’ve written apache upload progress module to have my lovely progress bar back again. As a lazy developer I’ve implemented reports in the same way as in nginx upload progress, so my applications are working without changing any signle line of code. If you were using nginx upload progress just drop the module, change your config and you’re good to go :)
I’m testing it in one of my production servers, but be carefull – it’s not well tested in other enviroments (i have gentoo with apache 2.2.8). Any feedback will be helpfull. Give me a note in comments if you encounter any problems.
So you want to be cool and have your own sexy progress bar in your app? Keep reading ;)
To install module you must download it using git:git clone git://github.com/drogus/apache-upload-progress-module.gitapxs2 -c -i -a mod_upload_progress.c- -c is for compiling
- -i is for installing (copy mod_upload_progress.so to apache library dir)
- -a is for activating (add LoadModule option into your apache conf file)
LoadModule upload_progress_module path/to/apache-upload-progress-module/.libs/mod_upload_progress.soUploadProgressSharedMemorySize 1024000This sets shared memory size to 1M. By default it’s 100kB.
To add tracking and reporting upload for a virtual host in apache you will need to add:
<Location />
# enable tracking uploads in /
TrackUploads On
</Location>
<Location /progress>
# enable upload progress reports in /progress
ReportUploads On
</Location>
Now all uploads will be tracked and reports are under /progress
Format of the report is JSON. From nginx wiki:
The returned document is a JSON text with the possible 4 results:
- the upload request hasn’t been registered yet or is unknown:
new Object({ ‘state’ : ‘starting’ })
- the upload request has ended:
new Object({ ‘state’ : ‘done’ })
- the upload request generated an HTTP error:
new Object({ ‘state’ : ‘error’, ‘status’ :
}) One error code that is interesting to track for clients is HTTP error 413 (Request entity too large)
- the upload request is in progress:
new Object({ ‘state’ : ‘uploading’, ‘received’ :
, ‘size’ : }) The HTTP request to this location must have either an X-Progress-ID parameter or X-Progress-ID HTTP header containing the unique identifier as specified in your upload/POST request to the relevant tracked zone. If you are using the X-Progress-ID as a query-string parameter, ensure it is the LAST argument in the URL.
Now the last thing to do is to implement progress bar. I don’t like repeating others and there is great tutorial on setting up upload progress bar with nginx and merb
UPDATE: I released jquery upload progress library with Safari 3 support. More info here. UPDATE2: I’ve upgraded prototype version to work in Safari.
It’s for merb and nginx but if you drop the scripts in your rails app and with apache-upload-progress-module it will work. :) Basically if you have your own code handling uploads (for example using attachment_fu) you can just add javascript and css – it’s unobtrusive.
If you’re using prototype I’ve rewritten script and made a demo. You can also grab files
I hope you enjoy this article. Progress bar is in my opinion one of the most useful technics – there is nothing more annoying than large file uploading without any info on state of an upload.
pimpmaster said
Jun 25, 2008 @ 09:23 PM
Nice writeup! :)
Your demo is busted in Safari 3 though (No progress is shown) :(
Micha said
Jun 26, 2008 @ 10:41 AM
For some reason, I get an error in my error_log: [error] (12)Cannot allocate memory: Upload Progress cache: could not create shared memory segment Configuration Failed
This error can be avoided by setting UploadProgressSharedMemorySize to something less than ~75000.
My Apache is Server version: Apache/2.2.8 (Unix) Server built: Mar 4 2008 21:37:02 on MacOS 10.5 (the builtin one) and was compiled with http://pastie.org/222492
Any thoughts?
Drogomir said
Jun 26, 2008 @ 11:16 AM
pimpmaster: I will check it :)
Micha: I don’t know what causes this problem but I changed default shared memory size to 50kB – it will be safer then 100kB. You can change shared memory size even to 10240 (10kB) if you don’t provide hundreds of uploads simultaneously it’s not a problem – it’s only for keeping upload data (key, upload size and size of received bytes). Upload data is removed from memory 60s after upload has ended.
Ziggy said
Jun 26, 2008 @ 09:02 PM
Does this work with https ? I can’t seem to get it working on FC8/x86_64, the /progress keeps reporting ‘starting’ .... httpd-2.2.8-1.fc8/php-5.2.4-3
Ziggy said
Jun 26, 2008 @ 09:59 PM
Doesn’t seem to matter if it’s https or not, the /progress is stuck on ‘starting’ anyways (FC8 etc.)
Drogomir said
Jun 27, 2008 @ 12:16 AM
Ziggy: Did you provide existing X-Progress-ID? It is in “starting” state if it can’t find valid upload data with given X-Progress-ID.
Ted said
Jun 27, 2008 @ 06:52 PM
Confirm not working in safari 3. Seems like a JS issue? =/
Drogomir said
Jun 28, 2008 @ 10:34 PM
I’ve checked it in safari on windows and it seems to be problem with sending ajax request while page is loading. I will investigate this one :)
Michele said
Jun 29, 2008 @ 01:02 AM
Yep, it definitely seems to be a problem with sending an AJAX request while loading a page (indeed, if you manually enter the AJAX call in the address bar prepending javascript: the call is executed correctly, but then it stops sending the main request).
I imagine it should be possible to fix this by using an iframe to separate the two requests…I’ll play with it tomorrow.
Meanwhile, thanks for the module: it has really helped! :)
Michele said
Jun 29, 2008 @ 12:36 PM
I managed to make it work in Safari, too: you need to insert an iframe in the page, give it a name (e.g. name=”progress”) and have it load a static HTML page which you’ve carefully crafted with: basic HTML structure (making sure you load all JS libraries you need) and in the body you define a function with the AJAX call and logic to update the progress bar. Of course, this function needs to passed the uuid.
Then from the main page you call name_given_to_the_iframe.name_given_to_the_function(uuid) In the function you can access the main page DOM object like so: parent.document.getElementById()
Drogomir said
Jun 29, 2008 @ 04:26 PM
Michele:
Thanks for checking it. I will update my example shortly :)
I will add Safari hacks to upload-progress script: http://github.com/drogus/jquery-upload-progress/tree/master and possibly make prototype version with some tweaks like in jQuery version.
Michele said
Jun 29, 2008 @ 04:57 PM
Just let me know if there’s anything not clear in my explanation. :)
Drogomir said
Jun 30, 2008 @ 03:03 AM
I’ve added safari support to jquery version of upload progress script: http://github.com/drogus/jquery-upload-progress/tree/master . Tomorrow I will try to code prototype version with callbacks and other useful stuff.
Check new demo which is supposed to work in safari: http://drogomir.com/files/blog/jquery-upload-progress/example
Matze said
Aug 20, 2008 @ 01:19 AM
Hi, i just installed upload_progress_bar and my Progressbar stopps counting after about 250000Bytes received. Upload still works and the status changed to done after finishing upload.
My Setup: Apache 2.2.9, mod_rails (Passenger) 2.0.3 and apache-mod-upload-progress (installed yesterday)
Any Ideas?
btw. thanks for the apache-mod!
Drogomir said
Aug 20, 2008 @ 07:20 AM
Matze:
Which browser did you use? While using Firefox 3 I have similar problems. Try to clear browser cache when the file is uploading and see if progress bar is still stuck. If yes, then it’s browser problem. If no, could you enable debug messages in apache and see if anything weird is reported in the log file?
Matze said
Aug 20, 2008 @ 01:13 PM
hmmm… indeed, it’s working on my Windows-Box with IE 7, FF2 and FF3. On my OS X (10.5.4) with FF3 and FF2 i had the above mentioned problem.
Sorry for the quick post without testing on an other box and thanks for your quick reply!
Maybe it’s a problem with caching, i will test it coming weekend.
Matze
OWL said
Aug 25, 2008 @ 01:51 AM
Hi, Thanks for module. Works fine with IE and FF but does not work with Opera. checked with opera = 9.51 under winxp.
Looks like Opera can’t do Ajax request during file upload. Does anybody know a hack for Opera ?
OWL said
Aug 25, 2008 @ 01:51 AM
Hi, Thanks for module. Works fine with IE and FF but does not work with Opera. checked with opera = 9.51 under winxp.
Looks like Opera can’t do Ajax request during file upload. Does anybody know a hack for Opera ?
Drogomir said
Aug 25, 2008 @ 10:50 AM
It works in Opera 9.5 for me (Ubuntu). I will check 9.51
If it can’t send ajax request it is the same issue that safari has.
Adam said
Aug 28, 2008 @ 04:33 AM
Hi There,Thanks for this patch. its awesome. Can anyone explain whether using this combined with mod_rails will get past the blocking issue with rails uploads? ie, will apache handle the upload and then pass it to rails or is rails handling the whole thing ??
Drogomir said
Aug 30, 2008 @ 01:17 PM
About the Opera: I’ve checked 9.52 and it is not working for me either. Hacks for safari are not working, so I will have to debug it on Opera.
Adam: mod_passenger is handling uploads – it’s faster and it doesn’t block rails instance. If you want to do something while uploading you must use Apache filters (like in my upload progress module)
Michael said
Sep 19, 2008 @ 07:12 PM
This is a great patch, but I’ve run into a problem. I had it installed, working beautifully (like I said, it’s great), very happy, etc, etc—then I installed an SSL cert. The httpd.conf directives in my Virtual Host blocks no longer were obeyed when viewing through HTTPS. I can move the directives outside a virtual host, but then your plugin becomes buggy: ’/progress’ works fine before you start an upload (state:starting), but when you try to upload a file, the upload script runs forever, as does the ’/progress’ feed.
I’m kind of clueless regarding SSL, so perhaps I’m missing something. Any suggestions?
Drogomir said
Sep 21, 2008 @ 02:49 PM
Michael: I will have to test plugin and scripts with ssl. However I don’t have much time right now, so I’m afraid that it can last some time. Could you pastie logs with SSL turned on and possibly log from safari error console?
KTU said
Sep 26, 2008 @ 08:39 PM
This looks very promising one but I haven’t been able to make it work.
I was looking progress bar with RAILS form page which could pop-up window for the progress bar and show percentage how upload is progressing. Is this possible with the current tool ?
Any change to get full working demo from somewhere ? Yes I downloaded examples but it wont work is complaining about various javascript errors etc…
Thank You
Drogomir said
Sep 27, 2008 @ 03:05 AM
Here is example in jquery: http://drogomir.com/files/blog/jquery-upload-progress/example/
Could you paste those errors?
mikehal said
Sep 28, 2008 @ 09:17 AM
I’m considering your module – appreciate some guidance before I jump in:
my setup: OS X.5 Apache 2 and php 5.2.current. No prototype or jquery or yahoo stuff. I would like to be able to upload the file from an iframe, pass the error data and image path back to parent.js via an object literal and have the js call for the upload progress data on a setinterval httprequest and modify the width of a div in the iframe accordingly.
what am I calling exacty? will this module work for me? do I need the passenger/rails stuff? what is apxs2? apxs compiles it OK so long as I install and configure manually it looks like it might work. not sure how to test it though.
thanks!
Drogomir said
Sep 28, 2008 @ 01:45 PM
mikehal:
If you don’t use jquery/prototype you can still use upload progress, but you will have to write your own code that will get progress from server (it is served as JSON). If you’re not sure how it should look like, you can check jquery/prototype upload progress plugin sources or even some js code written to show lighttpd upload progress (apache upload progress has the same response format so it will work :).
Scenerio that you have desribed seems to look ok, so this module will work for you :)
You don’t need rails/passenger stuff. It only requires apache :)
apxs2 is tool to compile modules for Apache2. As long as it compiles and works you can use apxs or even gcc (but you will need to pass some options then).
The easier way to test it is to set it for one of virtual hosts and run one of examples from jquery/prototype upload progress module :) They are in the packages. But testing it locally is a little bit hard, cause files are uploading so fast that you won’t notice progress, so I recommend doing it on some remote server. Check post about javascript plugins – there are links for examples and packages (examples are in packages :)
mikehal said
Sep 30, 2008 @ 04:19 AM
Drogomir:
Thanks for the guidance. I have it compiled and I manually put the .so in the libexec dir and added it to httpd.conf. I was running Apache2 out-of-the-box so at first I just tried:
http://localhost/progress?X-Progress-ID=123
but got a 404 until I added the vhost stuff. Then I got:
new Object({ ‘state’ : ‘starting’ })
which got me excited. After hooking it up (an upload form with a hidden element for the UID; httpRequest/progress loop via an onSubmit) I cannot get anything more back – always state:starting. testing both locally and remote.
the prototype-assisted demo is not doing any better. stumped for now. What am I missing?
will recompile with gcc etc.
Drogomir said
Sep 30, 2008 @ 02:14 PM
mikehal:
Could you pastie apache error log (with log leve debug) ? Which browser have you used for this tests? Try jquery or prototype demo on firefox with firebug and check requests in console.
mikehal said
Sep 30, 2008 @ 06:14 PM
Drogomir:
Same result FF3 and SAF3. for bothe remote and local access to the same files. your prototype demo is working remotely , just not locally. Here is the remote access debug log for my stuff followed by your prototype-assisted demo remote access followed by yours hit locally (it hangs)
[debug] mod_upload_progress.c(139): Upload Progress: Upload in trackable location: /processors/upload.php. [debug] mod_upload_progress.c(682): Upload Progress: Found id=f264a380e4e901e10b5e2399f209a472 in location with reports enables. uri=/progress [debug] mod_upload_progress.c(707): Node with id=f264a380e4e901e10b5e2399f209a472 not found for report
yours via remote
[debug] mod_upload_progress.c(139): Upload Progress: Upload in trackable location: /drogus/example/index.html. [debug] mod_upload_progress.c(143): Upload Progress: Progress id found: 494b24386a970469e70d4c70d418c6c4. [debug] mod_upload_progress.c(369): Upload Progress: Inserted node into an empty list. [debug] mod_upload_progress.c(462): Upload Progress: Added upload with id=494b24386a970469e70d4c70d418c6c4 to list. [debug] mod_upload_progress.c(682): Upload Progress: Found id=494b24386a970469e70d4c70d418c6c4 in location with reports enables. uri=/progress [debug] mod_upload_progress.c(697): Node with id=494b24386a970469e70d4c70d418c6c4 found for report
yours locally (same server, upload file etc)
[debug] mod_upload_progress.c(139): Upload Progress: Upload in trackable location: /drogus/example/index.html. [debug] mod_upload_progress.c(143): Upload Progress: Progress id found: fd8bdd71016085e791c0056e759e797f. [debug] mod_upload_progress.c(374): Upload Progress: Inserted node at the end of the list. [debug] mod_upload_progress.c(462): Upload Progress: Added upload with id=fd8bdd71016085e791c0056e759e797f to list.
Obviously if your prototype is working the module compiled OK and is loaded. I’m choking it with my script somehow.
mikehal said
Oct 01, 2008 @ 01:13 AM
Looks like assigning the uid via a hidden form field is a problem. (the PECL uploadprogress PHP ext assigns it this way).
Drogomir said
Oct 01, 2008 @ 12:54 PM
Unfortunately none of those upload progress modules (for nginx, lighttpd and mine for apache) gets UID from form fields. Currently you can only attach it to address as GET parameter or send in header X-Progress-ID. If PECL assigns it this way, I will consider adding possibility to POST parameters, but as I said before I don’t have much time, so I think that it won’t be quick…
Rick said
Oct 02, 2008 @ 02:31 AM
I successfully compiled this on OS X.5.5. However, apachectl does not like the resulting shared lib.
httpd: Syntax error on line 118 of /private/etc/apache2/httpd.conf: Cannot load /usr/libexec/apache2/mod_upload_progress.so into server: dlopen(/usr/libexec/apache2/mod_upload_progress.so, 10): no suitable image found. Did find:\n\t/usr/libexec/apache2/mod_upload_progress.so: mach-o, but wrong architecture
I am using the default Apache 2.2 on OS X with its apxs command. Any ideas?
mikehal said
Oct 02, 2008 @ 09:26 PM
Rick: I ran into the same problem with the same setup so I flagged apxs to compile only and moved the resulting .so from the .libs dir to /usr/libexec/apache2 by hand and added the load module directive to httpd.conf. It’s working! (don’t forget to add in the vhost directive per Drogomir’s guide).
Rick said
Oct 02, 2008 @ 10:08 PM
mikehal: Thanks for the suggestion. I did the same on my box and I am still getting the error. I will have to try this on one of my Linux boxes. OS X has not been working too well for me since the upgrade to 10.5.5.
mikehal said
Oct 03, 2008 @ 10:30 PM
Drogomir:
I’ve had some time to test out what’s coming back from /Progress. A couple of questions/observations:
1. It looks like all other kv’s are wiped when state hits ‘done’. ?
If the polling interval is too short – say sub 300ms then the received kv comes back irregularly – one or two reports early on a 1.5MB file then nothing until complete. At 500 it’s smooth. But small files upload before the second poll occurs and I want to get at size: or received even if I’m late but the properties are undefined.
2. Can Apache be directed [from a module] to call a specific js function? Just curious. Not exactly sure what I would do with it. (There’s a slight delay before /progress becomes available)
Drogomir said
Oct 04, 2008 @ 02:51 PM
What do you mean by kv?
Irrgularity of responses with short interval is caused by uploading file. Normally, response from server should be very quick (it’s small and generated in a few miliseconds), but while uploading, responses are much slower, so you have to wait longer. If you want to provide progress bar for smaller files you can consider swfupload – it looks great out of the box and you don’t have to fight with all those ajax issues. Ofcourse there are also bad sides: it is stuck at 0% on linux (and it’s very slow for me), it’s much more obtrusive and there is more people with disabled flash than people without javascript.
If it fits your needs you can give it a try.
2. Apache is server side and javascript is client side so basically it can’t call any javascript (there are server side js projects like jaxer but for keeping things simple let’s forget about it for now). I think that your question should be – Can apache force javascript to call a specific function?
The answer is: yes, it can. If apache will response with type ‘text/javascript’ it will be executed. But technically it’s pretty much the same as calling a function in js after getting JSON response.
The other way to do it (and it’s probably closer to what you described) is a push server. Rather than calling xhr requests with given interval server forces javascript to make a request.
mikehal said
Oct 05, 2008 @ 02:12 AM
By kv’s I meant the key-value pairs coming in from /progress. I guess I should say properties? When state:’done’ , size: and received: are no longer defined. I have to store them ahead of state:done which for a really fast upload is touchy. I’m new at this stuff so forgive me if this is idiotic. ;)
mikehal said
Oct 05, 2008 @ 02:29 AM
per swfupload: I dug into that and got it working-ish but it, well, led me to your module if you know what I mean. Tricked out OOP js libraries tied into css definitions hooked up to some actionscript complied into an invisible ‘movie’ hurt my head.
George Moschovitis said
Oct 05, 2008 @ 03:21 AM
Does this module actually save the upload to a tempfile?
Drogomir said
Oct 05, 2008 @ 04:40 PM
mikehal: I think it’s not idiotic, but I don’t understand your question :) Could you give some examples or explain again?
George Moschovitis: this module is something like filter working while data is uploading. It only gives you information about upload. After data is uploaded you have to handle it yourself.
Tim said
Oct 08, 2008 @ 06:56 PM
Rick, mikehal -
It seems that apsx on the mac only compiles for one architecture… i.e. not a universal binary. I got around the load error mentioned by explicitly compiling all architectures like so:
sudo apxs -c -i -Wc,-arch -Wc,ppc7400 -Wl,-arch -Wl,ppc7400 -Wc,-arch -Wc,ppc64 -Wl,-arch -Wl,ppc64 -Wc,-arch -Wc,x86_64 -Wl,-arch -Wl,x86_64 -Wc,-arch -Wc,i386 -Wl,-arch -Wl,i386 mod_upload_progress.c
Maybe this should be mentioned in the readme?
(... and, yeah, that is one fugly command line grin)
HTH, Tim
Tim said
Oct 10, 2008 @ 04:52 AM
Out of curiosity, how long does upload info persist after the upload completes?
Drogomir said
Oct 10, 2008 @ 04:25 PM
Tim:
I will add your findings to README file.
Upload info expires after 60 seconds, but is cleaned after another upload starts (I’ve done it in the simple way – nodes are checked for expiry only when new node has to be tracked).
Attila said
Oct 30, 2008 @ 02:51 AM
Tim,
you could still add the -a parameter as well, I did it that way and I have no problems with this so far on OS X. Thank you for sharing your solution, apparently it helped a lot. Also, thanks to Drogomir for this great software. I’ll test the hell out of it now :)
Tim said
Nov 10, 2008 @ 04:23 PM
Attila -
I did try with the -a flag and for some reason it only compiled the i386 part and for some reason (I am not entirely sure why…) Leopard’s apache expects the x86_64 code to be there as well.
Maybe a bug in apxs?
Cheers, Tim
odorryagrinty said
Nov 14, 2008 @ 07:28 AM
Hello! Good site, much useful
Yves said
Nov 24, 2008 @ 08:48 PM
FYI – it does not work with Debian Etch and Apache 2.2 MPM Worker (Threaded)
Forcing reload of web server (apache2)... waiting ..apache2: Syntax error on line 185 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/upload_progress.load: Can’t locate API module structure `upload_progress’ in file /usr/lib/apache2/modules/mod_upload_progress.so: /usr/lib/apache2/modules/mod_upload_progress.so: undefined symbol: upload_progress failed!
Yves said
Nov 24, 2008 @ 08:55 PM
Arghs… it works. Just do it this way on Debian.
Yves said
Nov 24, 2008 @ 11:41 PM
Just to get it right… the X-Progress-ID is generated randomly and prevents users from polling foreign upload progress?
Drogomir said
Nov 25, 2008 @ 02:52 AM
Yves: Yes, you’re right – you need X-Progress-ID to identify current upload.
knodi said
Nov 30, 2008 @ 01:54 PM
I have a problem that the done state is never received it goes straight from uploading to error even thou the upload is done properly the done state is never sent.
is there something I need to do to get the done state?
how does this work with multiple downloads at the same time/cluster environment?
Drogomir said
Dec 01, 2008 @ 07:10 PM
knodi: could you set log level to debug and pastie some of debug messages?
Done state should be set just after upload is complete, you don’t have to do anything.
What do you mean by “multiple downloads”?
knodi said
Dec 02, 2008 @ 10:51 AM
@Drogomir
I got the done state now that added the UploadProgressSharedMemorySize 1024000.
But there is a more sinister problem that I came across “* Unexpected error in Passenger: The HTTP client sent incomplete upload data.” error.
found more info on it here http://groups.google.com/group/phusion-passenger/browse_thread/thread/4f496e3fde297a48
don’t know if its passenger or UploadProgress. But it happens only with both are used together.
Knodi said
Dec 02, 2008 @ 10:53 AM
@Drogomir
by multiple downloads I mean in a apache clustered environment.
Knodi said
Dec 04, 2008 @ 06:37 PM
Ignore my post about ““* Unexpected error in Passenger: The HTTP client sent incomplete upload data.” This seems to be a problem with Passenger (mod_rails) only on OSX.
One more thing how does this work in a apache cluster environment. Work my front(load balancer) apache server be able to get the /progress or does the apache that the load balancer apache proxies the request the only apache process able to reply with the correct /progress info.
To visualize this: lets say i have 1 load balancer apache(Lets call it Apache A) that proxies request to 2 other apache processes lets call them Apache B and Apache C.
So when the upload file post comes in Apache A will hand the request of the Apache B after this happens when I do /progress Apache A hands the request to Apache C as Apache B is busy with the upload request. So this will cause an error as Apache C doesn’t know anything about the upload that going on.
So my question is this an fair interpretation of how upload progress mod will work?
Drogomir said
Dec 06, 2008 @ 02:33 PM
Knodi: Yes, unfortunately you’re right. Upload data is shared only between vhosts. In case of such configuration the easiest way is to use some client side upload progress (SWFUpload maybe).
erik said
Jan 08, 2009 @ 01:34 AM
I have created a screen cast which shows how to use this module to create a file upload progress bar : http://www.railsillustrated.com/screencast-file-uploads-progress-in-rails-passenger.html.
Drogomir said
Jan 09, 2009 @ 11:46 AM
erik: Great stuff!
I’ve added link to your screencast in README so people can roll out their own progress bar easier :)
Slobodan said
Jan 09, 2009 @ 09:55 PM
I have a problem where I keep getting glibc errors in the Apache logs. Almost certainly caused by mod_upload_progress, because if I disable it the errors don’t appear anymore.
After less than 10 uploads (submitted one after the other) the browser just won’t upload anything anymore and is stuck forever waiting for the response. Happens both with prototype and jquery scripts.
Drogomir said
Jan 11, 2009 @ 04:54 PM
Slobodan:
Could you paste your logs? And write some more details about your environment (apache ver, OS etc.)?
Logs with glibc errors and part while upload progress is stuck.
Slobodan said
Jan 12, 2009 @ 12:50 AM
Peter, I debugged it and think I figured out what it was. My Apache configuration (2.2, ArchLinux) was loading all sorts of modules, including mod_mem_cache. Now for some reason the call to cache_free in line 434 was always crashing. To reproduce, I uploaded a file, waited 60s for the node to expire and submitted another file, at which point the worker process crashed at the call to cache_free.
From the backtrace I could see that the call to cache_free was calling the function with the same name, but in the mod_mem_cache, instead of the local one in the mod_upload_progress. How this was possible I still have no idea (or maybe I didn’t understand the code enough). As soon as I disabled mod_mem_cache, the progress tracking was working like a breeze.
One more thing: shouldn’t the line 442 be something like “node = prev-> next”?
Drogomir said
Jan 13, 2009 @ 09:32 AM
Slobodan:
Thanks for the info. I’ll try to change mod upload to work well with mem_cache.
In case of 442 line: These to code blocks (from 431 and 430) only throw the node out of the list and set it to previous node. Actual fetching of next node is done in 447 with node_fetch. It can’t be done with node = node->next; beaceuse of using shared memory (upload info must be shared between all vhosts – actual upload and /progress request can be handled by not the same vhosts).
I should refactor this pieces ;-) It could be done better without using continue.
Ron Evans said
Feb 04, 2009 @ 12:10 AM
Peter, thanks for doing so much for the community. You certainly saved me a lot of time!
I have forked versions of both the apache-upload-progress-module and the jquery plugin, which support JSON-P for cross domain requests. You need this if you want to have a separate upload server from your web server, in order for browser security to be happy.
I submitted pull requests on github from my forks, which are at http://github.com/deadprogrammer/apache-upload-progress-module and http://github.com/deadprogrammer/jquery-upload-progress for anyone else who might be interested.
Anyhow, thanks again for your contributions!
rishav said
Feb 06, 2009 @ 12:23 PM
Hi,
it seems to work fine in FF3 but breaks in IE7 for most images, the response is usually an error, Any ideas why is it breaking? ( I am using prototype 1,6,0,1 for development and rails 2,2,2 )
Thanks in advance
bootdisk said
Feb 24, 2009 @ 03:37 PM
Hi there and sorry for my poor english, as I’m french I’m trying to have this nice little piece of code working (apache2 on debian 4.0), but I can’t seem to figure it out..
I see the page /progress?X-Progress-ID=* being called, in firebug, but it doesn’t return anything (not a 404 though)
when I add &callback=xxx to the url I get :
the size is right, but “received” and “speed” never get updated, even though the uploads eventually succeed. I’ve tried to adapt your prototype example but it doesn’t work either..
furthermore, every time I attempt an upload I’m getting “[notice] child pid * exit signal Segmentation fault (11)” in apache2/error.log
what can be happening ?
ernesto said
Feb 25, 2009 @ 02:05 PM
thanks very much!!!
andy said
Feb 26, 2009 @ 05:35 PM
I’m getting an error in the JSON response…
“Badly formed JSON string: ‘�\t`x�\t���\t(new Object({ \’state\’ : \’uploading\’, \’received\’ : 132786800, \’size\’ : 318575923, \’speed\’ : 66393400 }));\r\n’”
Note the garbled chars before ‘(new Object…’ , not sure if this is something I’ve done in config or what the issue is.
Any ideas?
Thanks
andy said
Feb 26, 2009 @ 06:10 PM
I got this to work by altering mod_upload_progress.c
commented out a few bits…
774 /* get the jsonp callback if any */ 775 //const char *jsonp = get_json_callback_param®; 776 777 // fix up response for jsonp request, if needed 778 //if (jsonp) { 779 // completed_response = apr_psprintf(r->pool, “%s(%s);\r\n”, jsonp, response); 780 //} else { 781 completed_response = apr_psprintf(r->pool, “%s\r\n”, response); 782 //}
not sure if this is a really bad idea or not, but it works for me now…
please advise
Drogomir said
Feb 27, 2009 @ 07:47 AM
Could paste the url which is generating such a response? (you can see it in firebug)
It seems that getting response with jsonp is trying to get some weird url, but I can’t recreate this error on my machines.
Drogomir said
Feb 27, 2009 @ 08:25 AM
andy:
Could you update to newest version and see if it works with those lines uncommented? I’ve changed one thing in this commit: http://github.com/drogus/apache-upload-progress-module/commit/f318ec5d571040dff3591a44393420dbd88e7282 b which I think could be the reason but I can’t really check it – on my servers both versions work.
andy said
Mar 02, 2009 @ 07:59 PM
Hi Drogomir,
I’ve updated my dev & staging environments with new code from github. In both cases it is now working as designed without any modification from me.
Thanks for the fast feedback.
FYI, this was Debian Lenny distro that had the problem.
Alexwebmaster said
Mar 03, 2009 @ 09:53 AM
Hello webmaster I would like to share with you a link to your site write me here preonrelt@mail.ru
Wang said
Apr 20, 2009 @ 08:48 AM
Hi Drogomir, Can this module be used in Apache 1.3? It seems that many Apache APIs have changed since.
Dave said
Apr 25, 2009 @ 07:45 PM
The nginx upload progess is working with passenger for me.
pimpmaster said
May 14, 2009 @ 01:03 AM
Any chance of an nginx version of this article, now that Passenger now runs on the world’s fastest server software?
Dario said
Jun 04, 2009 @ 06:45 AM
Hi
I’m trying to install the upload progress module but when I run:
apxs2 -c -i -a mod_upload_progress.c
I got:
apxs:Error: Activation failed for custom /etc/apache2/httpd.conf file.. apxs:Error: At least one `LoadModule’ directive already has to exist..
Any idea what’s my problem?
Thanks!
Dario said
Jun 04, 2009 @ 07:34 AM
My context:
Apache 2.2.9 passenger Ruby Enterprise
Piotr Sarnacki said
Jun 04, 2009 @ 12:02 PM
Dario: Could you try just compile module? apxs2 -c mod_upload_progress.c
It seems like apache has problems with activating module in apache conf so if mod will compile you can manually add it to apache.
Michael Hasenstein said
Jun 06, 2009 @ 12:54 PM
What is this “new Object({...})” syntax?
That is not valid JSON, and it’s useless. An object is just {...(members)...}. That implies the creation of a new Object.
100% Equivalent: new Object() {}
Besides, operators (like “new”) are NOT allowed in JSON, ONLY “data”, such as {} [] (object and array literals).
http://json.org/
Piotr Sarnacki said
Jun 08, 2009 @ 12:16 AM
Michael Hasenstein: If it’s useless for you just don’t use it or fork it and change the responses :)
I’ve copied this syntax from lighttpd and nginx modules to make apache’s upload progress module compatible with them. I will check JSON without “new Object”. If it doesn’t break existing apps I will probably change it. Thanks for info on this.
Michael Hasenstein said
Jun 08, 2009 @ 09:18 PM
Try http://www.jsonlint.com/ for JSON validation.
Example: new Object({ ‘state’ : ‘starting’ }) only parses if changed to { “state”: “starting” }
JSON data should NEVER be eval()-ed, because this is insecure – arbitrary code can be executed! It should be read through a JSON parser instead, which would raise a SyntaxError exeption if it encounters your non-JSON.
JSON is a pure data format, which also makes it portable – usable by other languages. If you include Javascript statements it will work only on insecure pages that use eval() to parse Javascript. Since I like your module but would never call eval() on JSON I have a great interest in this matter ;-)
Besides, I love Javascript – after treating it like any other scripting language for 10 years I finally saw the light with some help from Douglas Crockford (I also recommend crockford.com), who among many other things developed JSON in the first place, by the way.
Cheers, Michael
Charlie said
Jun 17, 2009 @ 06:17 PM
This is working great for me if I am not using a virtual host. As soon as I try to use it on a virtual host, though, it stops working (the progress bar never changes).
I have dropped the following into the virtual host’s configuration file, but so far no dice. The url returns the X-Progress-ID after submit, so I know the form is interacting with the script, but the progress bar never changes.
<location /> # enable tracking uploads in / TrackUploads On </location> <location /> # enable upload progress reports in /progress ReportUploads On </location>Does anyone have any suggestions?
Dinesh said
Jun 26, 2009 @ 01:09 PM
Hi,
Could I club the module with mod_porter ? Mod Porter allows Apache to take care of the uploads for me but I am not able to get the uploaded status. How could I make it work with mod porter ?
Piotr Sarnacki said
Jun 26, 2009 @ 05:14 PM
Dinesh:
Upload progress module was tested with porter and fixed with this commit: http://github.com/drogus/apache-upload-progress-module/commit/787b138a7ef812fae0694003a0d997907a0c7b65
Do you have latest version from github? Could you post your setup (apache version, distribution, mod_porter version)?
Brettflan said
Jul 11, 2009 @ 03:02 AM
I’ve gotten it working with the latest Opera (9.64). It requires adding another iframe and setting the form to target the iframe (<form>) instead of just targeting the main window. A bit of a strange workaround, but it does work. Of course, you then have to change the way it deals with upload completion since that now occurs within the iframe.
Alexander said
Jul 26, 2009 @ 06:59 AM
Hi, haven’t read any of the other comments but does this work with multipe uploads?
Piotr Sarnacki said
Jul 26, 2009 @ 05:11 PM
Alexander: Yes. If you’re writing javascript code yourself you just have to generate random id for each download. If you’re using jquery or prototype plugin it’s already set up :)
iUser59 said
Aug 03, 2009 @ 02:57 PM
How do you do to redirect after the upload process? Because my redirection is executed (I see that in the log) but in my browser, I return to the same page (index.rhtml)
Piotr Sarnacki said
Aug 03, 2009 @ 05:21 PM
iUser59: could you paste some more results? Log and html that you use will be useful
Boss Resurfacing said
Sep 07, 2009 @ 04:50 AM
Off topic – Help with PM? lost password Boss Resurfacing Boss Resurfacing
ejgwzbolzxmx said
Oct 06, 2009 @ 12:29 AM
mqcfdraemahc
tzangms said
Oct 18, 2009 @ 06:50 AM
Thanks, this apache module is great! :D
But in fact, I encounter a problem, if I run nginx as reverse proxy in front of apache, then the upload progress will be not working.
And I tried lots of nginx configuration about headers, but useless, any idea?
Piotr Sarnacki said
Oct 18, 2009 @ 02:22 PM
X-Progress-ID can be sent as as a header o parameter. There might be problems with sending it as a header, but I don’t think that that you must do anything special to make it work in latter case.
Could you post some more details? How do you send X-Progress-ID, what is your config or anything that can be useful in solving that problem.
Also there is a plugin for nginx as I wrote in my post so maybe you could try to use that in your setup?
tzangms said
Oct 18, 2009 @ 03:45 PM
I am running django with mod_wsgi in apache, and nginx as reverse proxy in front of apache.
new Object({ ‘state’ : ‘starting’ })and I just tested the nginx upload progress module 10 minutes ago. finally it works.
It looks like in my setup, should use nginx to track the upload.
Thanx for your help. :D
supagroova said
Nov 02, 2009 @ 05:35 PM
Hi there,
We’ve been debugging our server cluster for a client with help from the Phusion team and we’ve found that a rather serious bug is originating from the upload-progress module.
I’d like to send you our info and backtraces etc to help you diagnose the problem. It seems the contact link up the top of the site is not working though. Please feel free to contact me if you’d like and I’ll provide you with the details.
rizwan said
Dec 06, 2009 @ 02:07 PM
hi , let me know can i use this with php and what i need to do ? i had tried with this . i am using apache and php 5.3 .
please give me the solution. i shall be thank full to you
eagedodrids said
Dec 12, 2009 @ 12:52 AM
Cool issue, didn’t thought this was going to be so great when I saw the link.
сайт гей знакомств said
Dec 17, 2009 @ 04:23 PM
а все таки: благодарю.
RSS feed for comments on this post
Leave a Comment