developer toolbox » ruby » rails

http://appeddesign.com/rails_swfupload

SWFUpload is a small JavaScript/Flash library to get the best of both worlds. It features the great upload capabilities of Flash and the accessibility and ease of HTML/CSS. See it in action....

    * Upload multiple files at once by ctrl/shift-selecting in dialog
    * Javascript callbacks on all events
    * Get file information before upload starts
    * Style upload elements with XHTML and css
    * Display information while files are uploading using HTML
    * No page reloads necessary
    * Works on all platforms/browsers that has Flash support.
    * Degrades gracefully to normal HTML upload form if Flash or javascript is unavailable
    * Control filesize before upload starts
    * Only display chosen filetypes in dialog
    * Queue uploads, remove/add files before starting upload

File Upload Fu

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba$ git clone git://github.com/davidsouth/rails-swfupload.git

Initialized empty Git repository in /home/hernad/devel/git/redmine-1.bring.out.ba/rails-swfupload/.git/
remote: Counting objects: 217, done.
remote: Compressing objects: 100% (150/150), done.
remote: Total 217 (delta 46), reused 217 (delta 46)
Receiving objects: 100% (217/217), 162.14 KiB | 6 KiB/s, done.
Resolving deltas: 100% (46/46), done.

podesenje repos-a na git.bring.out.ba

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/gitosis-admin$ vi git*conf

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/gitosis-admin$ git-commit -a -m "rails-swfupload" 
Created commit 1c98f7f: rails-swfupload
 1 files changed, 10 insertions(+), 1 deletions(-)
hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/gitosis-admin$ git push
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 373 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
refs/heads/master: abd9fec001456596070019ad8be36ab6d0b262a2 -> 1c98f7f7c1150c6c58050857651d97ec5cc05bf3
To git@git.bring.out.ba:gitosis-admin.git
   abd9fec..1c98f7f  master -> master
WARNING:gitosis.gitweb.set_descriptions:Cannot find 'rails-swfupload' in '/srv/bring.out.ba/git/repositories'
WARNING:gitosis.gitweb.generate_projects_list:Cannot find 'rails-swfupload' in '/srv/bring.out.ba/git/repositories'

git bring.out.ba

[remote "b-out-ba"]
        url = git@git.bring.out.ba:rails-swfupload.git
        fetch = +refs/heads/*:refs/remotes/b-out-ba/*

[branch "master"]
        remote = origin
        merge = refs/heads/master

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/rails-swfupload$ git push b-out-ba master

Initialized empty Git repository in ./
Counting objects: 217, done.
Compressing objects: 100% (150/150), done.
Writing objects: 100% (217/217), 162.95 KiB, done.
Total 217 (delta 46), reused 217 (delta 46)
refs/heads/master: 0000000000000000000000000000000000000000 -> ee1fe540bb238c6c5f7e17167f912fb55e3f3ae7
To git@git.bring.out.ba:rails-swfupload.git
 * [new branch]      master -> master

http://git.bring.out.ba/gitweb.cgi?p=rails-swfupload.git;a=summary

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/rails-swfupload$ sudo apt-get install libfreeimage-dev

hernad@nmraka-1:~/devel/git/redmine-1.bring.out.ba/rails-swfupload$ sudo gem install image_science

Bulk updating Gem source index for: http://gems.rubyforge.org
Successfully installed rubyforge-1.0.0
Successfully installed hoe-1.5.3
Successfully installed RubyInline-3.6.7
Successfully installed image_science-1.1.3
4 gems installed
Installing ri documentation for rubyforge-1.0.0...
Installing ri documentation for hoe-1.5.3...
Installing ri documentation for RubyInline-3.6.7...
Installing ri documentation for image_science-1.1.3...
Installing RDoc documentation for rubyforge-1.0.0...
Installing RDoc documentation for hoe-1.5.3...
Installing RDoc documentation for RubyInline-3.6.7...
Installing RDoc documentation for image_science-1.1.3...

http://blog.vixiom.com/2006/09/08/multiple-file-upload-with-flash-and-ruby-on-rails/

http://blog.vixiom.com/2007/06/29/merb-on-air-drag-and-drop-multiple-file-upload/

Merb

Like Ruby on Rails, Merb (Mongrel + ERB) is an MVC framework. Unlike Rails, Merb is ORM-agnostic, JavaScript library agnostic, and template language agnostic, preferring plugins that add in support for a particular feature rather than trying to produce a monolithic library with everything in the core. In fact, this is a guiding principle of the project, which has led to third-party support for the ActiveRecord, DataMapper, and Sequel ORMs.

In addition, it means that the core code in Merb is kept simple and well organised. This has multiple benefits. It means it‘s faster for one thing. It‘s also easier to understand, maintain and extend.

http://jimneath.org/2008/05/15/swfupload-paperclip-and-ruby-on-rails/

http://www.thoughtbot.com/projects/paperclip

Paperclip is a plugin for Ruby on Rails’ ActiveRecord that lets files work as simply as any other attributes do. There are no extra database tables, only one library to install for image processing, and the ease of being able to refer to your files as easily as you refer to your other attributes.
Getting it

The main repository is at

git://github.com/thoughtbot/paperclip.git

The SVN mirror is

https://svn.thoughtbot.com/plugins/paperclip/trunk/

There is a gem version available on RubyForge at

http://rubyforge.org/projects/paperclip/

Releases are tagged on both git and svn, and gems are made from tagged releases. The svn history is not as detailed as the git history, so if you need to see specific revisions, check the git logs (which you can do on github, if you’re using svn).
Usage

In your model:

class User < ActiveRecord::Base
has_attached_file :avatar,
:styles => { :medium => "300x300>",
:thumb => "100x100>" }
end

In your edit and new views:

<% form_for :user, :html => { :multipart => true } do |form| >
<= form.file_field :avatar >
< end %>

In your controller:

def create
@user = User.create( params[:user] )
end

In your show view:

<%= image_tag @user.avatar.url >
<= image_tag @user.avatar.url(:medium) >
<= image_tag @user.avatar.url(:thumb) %>

Bugs and Features

Please use our Lighthouse account for reporting bugs. You can post to the Google Group if you want discuss a feature. Or you could use either for either; what’s important is that we hear from you.
Requirements

Only ImageMagick, which you can install quite easily install via apt, yum, or port, or the package manager of your choice.
More?

You may want to check out the Paperclip rdocs
Copyright and License

Paperclip is © 2008, thoughtbot, inc. and is released under the MIT License.

http://www.liverail.net/railsdav

Scalable File Uploads with Merb¶

For a new application I'm working on we need to upload some large files to our web application. Also because the files are both large and in processing them there are many sql insert and updates that follow the whole process is pretty intensive taking roughly 1 to 2 minutes a file. I've been reading about merb for a few months and off and on created small projects with it hoping to one day put it to use. Tonight I was pleased that it really does make file uploading easy and appears fast. Here's a quick overview of what I did to get it setup.

sudo gem install merb -y

Create the basic merb project

merb -g uploader

Setup the upload controller and view

cd uploader/dist
mkdir app/views/uploader
touch app/controllers/uploader.rb
touch app/views/uploader/index.rhtml

We'll have two actions defined in the Uploader controller:

• index, to display the form

• upload, to handle the file upload post

class Uploader < Application # http://www.cs.tut.fi/~jkorpela/forms/file.html
def index
render
end

def upload
                puts params[:file].inspect
                FileUtils.mv params[:file][:tempfile].path, MERB_ROOT+"/uploads/#{params[:file][:filename]}" 
                redirect "/uploader" 
        end
end

Define the form view

<p>Upload a new file</p>
<form action="/uploader/upload" method="post" enctype="multipart/form-data">
<fieldset>
<input type="file" name="file" size="80"/>
<input type="submit" value="Upload"/>
</fieldset>
</form>

Finally, start your merb application up (from the uploader folder not instead the dist directory).

merb

You might need to disable sql_session in your dist/conf/merb.yml

change:

:sql_session: true

to:

:sql_session: false

The server is running on port 4000 and the request path is /uploader e.g. http://localhost:4000/uploader

Next I'd like to work more tightly integrating this with my rails application. A few things I want to focus on:

• the traffic is routed in my nginx setup.
• whether or not the merb process should do the long processing needed after the file is uploaded.
• how to get the results of the processed xml files back into my database for my rails app to read from.
• how to indicate to the users that the file has been uploaded but may still be being processed.

http://www.captain.at/ajax-file-upload.php

FUD revisited (1): Can you upload files with AJAX?¶

Posted by th0fu on September 30th, 2005

The short answer is: No.

The long answer is: You can’t, because AJAX get/post data is gathered via JavaScript, and JavaScript has no way at getting at local file contents (for security reasons).

But you can use AJAX to get information on the running upload, as long as your framework has functions that give information on current uploads; and your webserver doesn’t buffer upload data.

In Ruby on Rails that works like this: If you enable the UploadProgress module, Rails stores information about the progress in the session, so while the upload is doing its thing, you can poll the server via an AJAX call and get nice status information. The Upload itself is made via a “traditional” HTML upload form to a hidden IFRAME (the helpers will do all this hard work for you transparently!).

See Sean Treadway’s demo and articles about this.

hernad@nmraka-1:~$ sudo gem install mongrel_upload_progress

Bulk updating Gem source index for: http://gems.rubyforge.org
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...
Successfully installed gem_plugin-0.2.3
Successfully installed daemons-1.0.10
Successfully installed fastthread-1.0.1
Successfully installed cgi_multipart_eof_fix-2.5.0
Successfully installed mongrel-1.1.5
Successfully installed mongrel_upload_progress-0.2.2
6 gems installed
Installing ri documentation for gem_plugin-0.2.3...
Installing ri documentation for daemons-1.0.10...
Installing ri documentation for fastthread-1.0.1...

No definition for dummy_dump

No definition for dummy_dump

No definition for rb_queue_marshal_load

No definition for rb_queue_marshal_dump
Installing ri documentation for cgi_multipart_eof_fix-2.5.0...
Installing ri documentation for mongrel-1.1.5...
Installing ri documentation for mongrel_upload_progress-0.2.2...
Installing RDoc documentation for gem_plugin-0.2.3...
Installing RDoc documentation for daemons-1.0.10...
Installing RDoc documentation for fastthread-1.0.1...

No definition for dummy_dump

No definition for dummy_dump

No definition for rb_queue_marshal_load

No definition for rb_queue_marshal_dump
Installing RDoc documentation for cgi_multipart_eof_fix-2.5.0...
Installing RDoc documentation for mongrel-1.1.5...
Installing RDoc documentation for mongrel_upload_progress-0.2.2...

Upload Progress Bars by way of Streaming Javascript¶

I implemented the XUpload technique of web upload progress bars by streaming javascript (instead of using polling AJAX calls) in Rails. I did this by extending the existing mongrel upload progress plug-in with a new one.

mongrel_streaming_upload_progress-0.1.gem

You'll find an example directory in there with a small Rails application. Start the example with this command

mongrel_rails start -p 3000 -S config/mongrel_upload_progress.conf

and navigate to http://localhost/files/new in Firefox.

The technique is to use onsubmit to load an iFrame which gets a stream of <script> elements telling the page how to update the progress bar.

Questions

1. Why is this better?
      Because the traditional method is to have the client browser send a question "how much has been uploaded?" every 3 seconds to the web-server only to get the response "12% has been uploaded." Instead the XUpload technique just streams, in one response, "you've uploaded 5%... you've uploaded 8%... you've uploaded 9%." This is less stress on the server and client.
   2. Why not use an AJAX request and eval the response instead of this iFrame hack?
      Because (prototype) AJAX requests will not eval the response one statement at a time. (Does anyone have a library that can do that?)
   3. Does it work on Safari?
      Not yet, but it should; I can't figure out why. XUpload does.
   4. Why the other iFrame that the form is targeted to?
      Because if I don't have that, the second (important) iFrame does not get updated with the onsubmit. (I guess this is because the browser knows it's going to a new page, so won't load new frames.)

hernad@nmraka-1:~/Desktop$ sudo gem install mongrel_streaming_upload_progress-0.1.gem

Successfully installed mongrel_streaming_upload_progress-0.1
1 gem installed
Installing ri documentation for mongrel_streaming_upload_progress-0.1...
Installing RDoc documentation for mongrel_streaming_upload_progress-0.1...

Tramline

Product info

Tramline is an upload and download accelerator that plugs into Apache, using mod_python. Its aim is to make downloading and uploading large media to an application server easy and fast, without overloading the application server with large amounts of binary data.

Tramline integrates into Apache using mod_python. The application server is assumed to sit behind Apache, for instance hooked up using mod_proxy or mod_rewrite.

Tramline takes over uploading and downloading files, handling these within Apache. Only a small configuration change in Apache should be necessary to enable Tramline.

The application server remains in complete control over security, page and form rendering, and everything else. Minimal changes are necessary to any application to enable it to work with Tramline; in fact it's just setting two response headers in a few places in the code.

http://four.livejournal.com/tag/upload+progress+bar

How to test out the streaming upload progress bar. (It is assumed you have Rails already installed.)

• Starting Mongrel listening at 0.0.0.0:3000
  Link 1 comment|Leave a comment

mongrel_streaming_upload_progress-0.2 [Jul. 8th, 2007|03:04 am]
[ Tags | projects, upload progress bar ]

I made a little javascript class (which depends on prototype.js) that can interpret a stream of JSON objects separated by semicolons.

As an application of this, this Mongrel plug-in streams upload status when given a qualified upload_id. It looks like this:

    {"received":5939792,"status":200,"state":"uploading","size":36872784};
    {"received":6398544,"status":200,"state":"uploading","size":36872784};
    [...]
    {"received":36479568,"status":200,"state":"uploading","size":36872784};
    {"received":36856400,"status":200,"state":"uploading","size":36872784};
    {"received":36872784,"status":200,"state":"done","size":36872784};

(The format of these JSON objects is the same that is being used in the upload progress plug-in for Lighttpd 1.5 as described here)

Included with this package is the javascript class Ajax.Pull and an example Rails application.

mongrel_streaming_upload_progress-0.2.gem
mongrel_streaming_upload_progress-0.2.tgz

This should work with Safari 3 and Firefox. I haven't tested IE. When I get around to it will add a hack to Ajax.Pull which falls back on polling a URL for the JSON updates if the browser does not support Ajax streaming. This plug-in already supports this sort of polling with the ?single URL parameter.

http://orbanbotond.blogspot.com/2008/02/upload-progress-bar-by-streaming-json.html

Have you ever dealt with the problem to make an upload progress bar?
If you did, then it's time to continue reading.

I wanted to make one using javascript.
Basically there are two alternatives:

• The simpler one is to make periodic ajax requests to a url. That url must return the received_size/ total_size pair. All you need to do is to update your progress bar in the browser. This solution works in all those browsers which support AJAX. The disadvantage is that usually the browser will receive the (received_size/totoal_size) data pair delayed. So the progress bar won't show you the real progress.
• The second option would be to use streaming. Instead of polling periodically the server, we should make one single request, and the server would stream the receive_size/total_pair data back to the browser.

I like this streaming alternative much more because it gives you a better UI experience.
Let me give you some hints on the streaming alternative.

• The simplest and the secure way is to stream sequences of '' back to the browser. The returned javascript will be automatically executed in IE 6.0, Firefox, Safari and Opera. You can find a solution for this at Ry Dahl's article. However it works in a lot of browsers it's not the most elegant solution because you need to hardcode updateProgressBar in the server configuration file.
• A little bit more elegant solution is to stream JSON expression like {"total_size": 10, "received_size": 0}; {"total_size": 10, "received_size": 1}; {"total_size": 10, "received_size": 2}; ...

If you are streaming JSON expressions you need to write a javascript function which updates the progress bar according to the last JSON slice. You can do this using this javascript utility. This script periodically checks the received content and calls your updateProgressBar function passing the last received JSON slice to it. Unfortunately this script doesn't works with the latest prototype library.

The shortest and the most elegant alternative is to use the "onInteractive" AJAX callback function. By using this callback we don't need to periodically check the response. It is called every time when new data arrives to the browser.
Look at this beautifull code:
new Ajax.Request('/upload/progress', {
pos :0,
onInteractive: function(obj) {
var slice = obj['responseText'].slice(this.pos)
var jsonFormat = slice.substring(0,slice.length - 2);
var json = jsonFormat.evalJSON();
updataProgressBar(json.totalSize, json.receivedSize);
this.pos = obj['responseText'].length + 1;
},
});

Unfortunately this callback doesn't work in Opera neither in IE.
And I didn't find any w3 recommendation for enforcing the browsers to implement it. :(

Let me summarize the opportunities and how are they supported on different browsers:

                                       Opera latest    Firefox 2.0.0.2    Webkit    Safari    IE 6    IE 7
Periodic Ajax calls                            ok    ok    ok    ok    ok    ok
Streaming <script> tags                            ok    ok    ok    ok    ok    ok
Streaming JSON, using Ry's javascript            ok    ok    ok    ok    not    not
Streaming JSON, using onInteraction            not     ok    ok    not    not    not

So I need to keep up hacking till AJAX standard is born and implemented... :))

http://www.sibsoft.net/xupload.html

panda merb upload + video streaming + amazon s3 + ec2