Rails core - Aug 2012 - Automatic asset paths prevent logical asset organization

In 2fe70c1 (last year), rails changed from a static list of assets.paths to
automatically register any path under assets/*. (This was not reflected in Rails
Guides so I pushed an update to docrails.)

However, in my opinion, these automatic asset paths create confusion. 

The result is that the first-level directories under `(app|lib|vendor)/assets`
get swallowed and if I put assets into a directory of my own naming, it will
conflict with assets from another asset path. e.g.

    app/assets/images/icons/close.png

    vendor/assets/xyz/icons/close.png

Here, the path segments "images" and "xyz" become asset path
roots, so both of the above resolve as icons/close.png, and serve from
/assets/icons/close.png. (If you didn''t expect that, I should point out
that the file from app will take precedence.)

I don''t know if I''m missing how this is expected to be used. I
might expect app/assets/stylesheets, lib/assets/stylesheets, and
vendor/assets/stylesheets to correspond and mask each other, but not a
subdirectory of my own naming.

Take for example a typical frontend widget I might want to download and install:

    foo/widget.js
    foo/styles.css
    foo/images/click.png

If I want to keep this in a namespaced asset bundle instead of reorganizing it
into separate javascripts/stylesheets/images directories, I can''t just
drop the folder into vendor/assets, as the asset paths will swallow the
"foo" directory name, leaving the files naked in public/assets. In my
manifests, these also automatically become available directly as
"styles.css" and "widget.js" without a "foo/"
prefix.

I could arbitrarily put it into vendor/assets/javascripts/foo (or
vendor/assets/whatever/foo), but that makes no sense: it''s not all just
javascripts, it''s a whole package. Is it really the right solution to
just nest things 2 levels down so as not to lose the subdirectory for
namespacing?

For vendor/assets packages, ideally I think the way to handle such packages is
to define a "foo.js" and "foo.css" manifest to wrap the
vendored package, and keep all its files scoped under its own directory, e.g.

    vendor/assets/foo.css             # manifest file: //= require
''./foo/widget.css''
    vendor/assets/foo.js              # manifest file: //= require
''./foo/widget.js''
    vendor/assets/foo/widget.js
    vendor/assets/foo/styles.css
    vendor/assets/foo/images/click.png

(Alternately, the manifests could be index files inside the folder.)

These would correspond to the public paths:

    /assets/foo/styles.css  # not used directly when included through a manifest
    /assets/foo/widget.js   # not used directly when included through a manifest
    /assets/foo/images/click.png

This would make the package available in application.(css|js) as a simple
"require foo"

How best to configure rails for this? Should
assets/(stylesheets|javascripts|images) be magically considered
"global" or should the initializer for sub-paths under assets be
dropped altogether, leaving it to the user to organize assets how they see fit?

(Also, currently there''s that somewhat confusing mismatch between
app/assets and public/assets, where app/assets is organized into subdirectories
for javascripts & stylesheets, and public is not -- defaulting to just
assets as the config path instead of assets/* could fix that.)

Are there any established usage patterns I''m missing, or should Rails
consider something to organize this better by default?

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/dWp6EGpUJZUJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Hey Andrew,

I have always liked the automatic asset paths and thought they allowed the top
level Rails applications as well as engine authors the ability to create
namespaces via directories.

I read your whole email but kind of got lost. Is your main issue that creating a
namespace via directories is a chore since it has to be done in 3 places?

Or a more concrete question. I am the author of the less-rails and the matching
less-rails-bootstrap gems. I was very conscious about directory namespaces and
made sure that my namespace for less-rails-bootstrap was
"twitter/bootstrap". I am a firm believer that engines that deliver
assets should follow this pattern. So the question, what would you suggest doing
differently vs this setup?

https://github.com/metaskills/less-rails-bootstrap/tree/master/vendor/assets


 - Ken

On Aug 4, 2012, at 12:50 PM, Andrew Vit wrote:
> In 2fe70c1 (last year), rails changed from a static list of assets.paths to
automatically register any path under assets/*. (This was not reflected in Rails
Guides so I pushed an update to docrails.)
> 
> However, in my opinion, these automatic asset paths create confusion. 
> 
> The result is that the first-level directories under
`(app|lib|vendor)/assets` get swallowed and if I put assets into a directory of
my own naming, it will conflict with assets from another asset path. e.g.
> 
>    app/assets/images/icons/close.png
> 
>    vendor/assets/xyz/icons/close.png
> 
> Here, the path segments "images" and "xyz" become asset
path roots, so both of the above resolve as icons/close.png, and serve from
/assets/icons/close.png. (If you didn''t expect that, I should point out
that the file from app will take precedence.)
> 
> I don''t know if I''m missing how this is expected to be
used. I might expect app/assets/stylesheets, lib/assets/stylesheets, and
vendor/assets/stylesheets to correspond and mask each other, but not a
subdirectory of my own naming.
> 
> Take for example a typical frontend widget I might want to download and
install:
> 
>    foo/widget.js
>    foo/styles.css
>    foo/images/click.png
> 
> If I want to keep this in a namespaced asset bundle instead of reorganizing
it into separate javascripts/stylesheets/images directories, I can''t
just drop the folder into vendor/assets, as the asset paths will swallow the
"foo" directory name, leaving the files naked in public/assets. In my
manifests, these also automatically become available directly as
"styles.css" and "widget.js" without a "foo/"
prefix.
> 
> I could arbitrarily put it into vendor/assets/javascripts/foo (or
vendor/assets/whatever/foo), but that makes no sense: it''s not all just
javascripts, it''s a whole package. Is it really the right solution to
just nest things 2 levels down so as not to lose the subdirectory for
namespacing?
> 
> For vendor/assets packages, ideally I think the way to handle such packages
is to define a "foo.js" and "foo.css" manifest to wrap the
vendored package, and keep all its files scoped under its own directory, e.g.
> 
>    vendor/assets/foo.css             # manifest file: //= require
''./foo/widget.css''
>    vendor/assets/foo.js              # manifest file: //= require
''./foo/widget.js''
>    vendor/assets/foo/widget.js
>    vendor/assets/foo/styles.css
>    vendor/assets/foo/images/click.png
> 
> (Alternately, the manifests could be index files inside the folder.)
> 
> These would correspond to the public paths:
> 
>    /assets/foo/styles.css  # not used directly when included through a
manifest
>    /assets/foo/widget.js   # not used directly when included through a
manifest
>    /assets/foo/images/click.png
> 
> This would make the package available in application.(css|js) as a simple
"require foo"
> 
> How best to configure rails for this? Should
assets/(stylesheets|javascripts|images) be magically considered
"global" or should the initializer for sub-paths under assets be
dropped altogether, leaving it to the user to organize assets how they see fit?
> 
> (Also, currently there''s that somewhat confusing mismatch between
app/assets and public/assets, where app/assets is organized into subdirectories
for javascripts & stylesheets, and public is not -- defaulting to just
assets as the config path instead of assets/* could fix that.)
> 
> Are there any established usage patterns I''m missing, or should
Rails consider something to organize this better by default?
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
> To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/dWp6EGpUJZUJ.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Thanks Ken, I''ll try to be more clear:

Asset gems often exist just to isolate those resources into a simple 
package to save everyone the trouble of reorganizing third-party files 
specially for the asset pipeline & maintaining that reorganization going 
forward. For some things the reorganization makes sense, and I won''t 
disagree with the way you''ve packaged the less-bootstrap gem. +1 for 
namespacing under "twitter/bootstrap", but this isn''t
necessarily about
that.

The issue is that for drop-in use of third-party assets (from plain 
folders, not gems), I think it''s more logical to keep their files in
one
self-contained folder and namespace, without having to tease out and 
reorganize them. As a benefit, we can use a git submodule to update their 
whole package, and encapsulate access into it  using manifest files.

I think I''ll end up settling on something like this:

    vendor/assets/images/      # empty
    vendor/assets/javascripts/ # empty
    vendor/assets/stylesheets/ # empty
    vendor/assets/packages/xyz/
    vendor/assets/packages/xyz/random.js
    vendor/assets/packages/xyz.js # manifest //= ./xyz/random.js

Using "packages" is there to nest everything one level down so I can 
isolate the "xyz" package into its own directory.

There doesn''t seem to be any documentation or conventions for this so I
wanted to open the discussion around how third-party assets should be 
handled for drop-in use.

My original point was that adding another folder of my own naming e.g. 
"assets/xyz" would shadow the standard folders: I don''t think
that''s
desired behaviour. I did update docrails to reflect how it currently works, 
but let''s discuss if the docs were originally right and Rails currently
isn''t: https://github.com/lifo/docrails/commit/ad088f09

So, to be more concrete:

1. Should asset paths be limited to (images|javascripts|stylesheets) 
instead of * glob to prevent accidental shadowing? (Other asset paths can 
be added explicitly.)

2. If assets/* should remain, should Guides document how other directories 
in there are meant to be used? (e.g. just for adding extra file types like 
"flash")

3. Promote something like "vendor/assets/packages" as an optional 
convention, or bless it as a standard path for drop-in use of asset folders?

Andrew

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/hNt4wQaBYZEJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

I understand what you mean. For my gem oojspec and oojs I use the 
manifest approach you have described:

https://github.com/rosenfeld/oojspec/blob/master/app/views/oojspec/runner.html.erb#L8
https://github.com/rosenfeld/oojspec/blob/master/lib/assets/javascripts/oojspec.js.coffee
https://github.com/rosenfeld/oojspec/blob/master/vendor/assets/javascripts/buster/all.js.coffee

You''ll notice I have submodule to third-party libraries (Buster.js):

https://github.com/rosenfeld/oojspec/tree/master/external/busterjs

Then I symlink the ones I''m interested in here (yes, I know this
won''t
work on Windows, but I don''t think I would get any contributions from 
Windows users anyway):

https://github.com/rosenfeld/oojspec/tree/master/vendor/assets/javascripts/buster


The same happens with oojs. Submodules:

https://github.com/rosenfeld/oojs/tree/master/resources

Linked assets:

https://github.com/rosenfeld/oojs/tree/master/vendor/assets/javascripts

Then the custom assets go into the lib directory instead of the vendor one:

https://github.com/rosenfeld/oojs/blob/master/lib/assets/javascripts/oojspec_helpers.js.coffee


But I do share your feeling that it should be easier to use 
third-library JavaScripts in Rails.

Maybe we could support something like AssetsDependencies similar to a 
Gemfile where you would specify the url to the assets or their 
repositories and what files should be required and in which order (or 
specify their dependencies) so that we could run some rake task to 
download them and write the manifest files. In that case we could add 
those files to the VCS or just do that dynamically before deploying like 
it happens with "rake assets:precompile".

I''d love to see something like that for Rails. Currently whenever I
want
to use some library with Rails I have to write my own gem when I don''t 
find one, as it happened with this one:

https://github.com/rosenfeld/jquery-layout-rails

What do you think about the idea of Rails providing something like the 
AssetsDependencies DSL and a rake task as mentioned above?

Best,
Rodrigo.


Em 04-08-2012 13:50, Andrew Vit escreveu:
> In 2fe70c1 (last year), rails changed from a static list of assets.paths to
automatically register any path under assets/*. (This was not reflected in Rails
Guides so I pushed an update to docrails.)
>
> However, in my opinion, these automatic asset paths create confusion.
>
> The result is that the first-level directories under
`(app|lib|vendor)/assets` get swallowed and if I put assets into a directory of
my own naming, it will conflict with assets from another asset path. e.g.
>
>      app/assets/images/icons/close.png
>
>      vendor/assets/xyz/icons/close.png
>
> Here, the path segments "images" and "xyz" become asset
path roots, so both of the above resolve as icons/close.png, and serve from
/assets/icons/close.png. (If you didn''t expect that, I should point out
that the file from app will take precedence.)
>
> I don''t know if I''m missing how this is expected to be
used. I might expect app/assets/stylesheets, lib/assets/stylesheets, and
vendor/assets/stylesheets to correspond and mask each other, but not a
subdirectory of my own naming.
>
> Take for example a typical frontend widget I might want to download and
install:
>
>      foo/widget.js
>      foo/styles.css
>      foo/images/click.png
>
> If I want to keep this in a namespaced asset bundle instead of reorganizing
it into separate javascripts/stylesheets/images directories, I can''t
just drop the folder into vendor/assets, as the asset paths will swallow the
"foo" directory name, leaving the files naked in public/assets. In my
manifests, these also automatically become available directly as
"styles.css" and "widget.js" without a "foo/"
prefix.
>
> I could arbitrarily put it into vendor/assets/javascripts/foo (or
vendor/assets/whatever/foo), but that makes no sense: it''s not all just
javascripts, it''s a whole package. Is it really the right solution to
just nest things 2 levels down so as not to lose the subdirectory for
namespacing?
>
> For vendor/assets packages, ideally I think the way to handle such packages
is to define a "foo.js" and "foo.css" manifest to wrap the
vendored package, and keep all its files scoped under its own directory, e.g.
>
>      vendor/assets/foo.css             # manifest file: //= require
''./foo/widget.css''
>      vendor/assets/foo.js              # manifest file: //= require
''./foo/widget.js''
>      vendor/assets/foo/widget.js
>      vendor/assets/foo/styles.css
>      vendor/assets/foo/images/click.png
>
> (Alternately, the manifests could be index files inside the folder.)
>
> These would correspond to the public paths:
>
>      /assets/foo/styles.css  # not used directly when included through a
manifest
>      /assets/foo/widget.js   # not used directly when included through a
manifest
>      /assets/foo/images/click.png
>
> This would make the package available in application.(css|js) as a simple
"require foo"
>
> How best to configure rails for this? Should
assets/(stylesheets|javascripts|images) be magically considered
"global" or should the initializer for sub-paths under assets be
dropped altogether, leaving it to the user to organize assets how they see fit?
>
> (Also, currently there''s that somewhat confusing mismatch between
app/assets and public/assets, where app/assets is organized into subdirectories
for javascripts&  stylesheets, and public is not -- defaulting to just
assets as the config path instead of assets/* could fix that.)
>
> Are there any established usage patterns I''m missing, or should
Rails consider something to organize this better by default?
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Rodrigo, your symlink strategy is yet another interesting idea...

But I don''t think reinventing Bundler for javascripts is really a
solution
to the problem. You can still wrap it in a gem and use Bundler for that.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/ZSNkTsEs7kwJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Em 05-08-2012 18:07, Andrew Vit escreveu:
> Rodrigo, your symlink strategy is yet another interesting idea...
>
> But I don''t think reinventing Bundler for javascripts is really a 
> solution to the problem. You can still wrap it in a gem and use 
> Bundler for that.
This isn''t about reinventing Bundler as they would have quite different
goals and strategy.

I already currently wrap my JS libraries/frameworks in gems when they 
don''t already exist but replicating every JS library as a gem
containing
just the assets just doesn''t feel right to me.

Also, while I spend about 10 minutes to vendor jQuery-layout as a gem, I 
think it could take a single minute to do the same through a DSL like 
I''ve described. Upgrading the library version would be even faster with
the DSL approach, specially if you''re the gem''s author.

Think about this. Suppose there are a thousand interesting JS 
libraries/frameworks out there that it would worth to package as a gem.

Then suppose there are 5 Ruby web frameworks willing to extend their 
frameworks with plugins packaged as a gem and all of them decide to 
package the assets as a gem. Then you could possibly have 5,000 gems in 
the Ruby world that actually contain no Ruby code at all, except for the 
plugin glue code that could be different for each web framework.

All of that just to make it easier for users to use a JS library/framework.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Monday, August 6, 2012 5:23:46 AM UTC-7, Rodrigo Rosenfeld Rosas
wrote:
>
> Then you could possibly have 5,000 gems in 
> the Ruby world that actually contain no Ruby code at all, except for the 
> plugin glue code that could be different for each web framework. 
>
> All of that just to make it easier for users to use a JS 
> library/framework. 
>
+1, I see this as a mild abuse of rubygems. There are already naming 
conflicts. 

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/Swvjzocjq7wJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.