Ironruby core - Jul 2009 - [IronPython] [Code Review] "XAP-less" Silverlight application support

Michael,

You just had to ask now =P ... I was going to blog about this more extensively
tomorrow, but here''s a preview of my thinking about this:

My motivation behind xap-less is to make DLR Silverlight apps less complex, and
more like how the web works. This change is just an experiment in that
direction, and I''m very open to any opinions folks have about this.

First things first though, this change make xap-less completely opt-in ... you
have to use the "downloadScripts=true" InitParams setting to enable
it. Without this param it will behave as it used to. Also, even with this
setting it will not download the file if it exists in the XAP. I was very
conservative about this change, as I''m just playing around with some
ideas.

Also, even if xap-less becomes the default, I''d make sure existing
deployments did not break; in the xap will always be an option for deployment
(and quite possibly the preferred way), but I want to get back to the place
where we didn''t need a XAP file and didn''t need Chiron for
development.

Eh, I know that was the messaging around XAP files being compressed and faster
for download times, but that''s because they were synchronously
downloaded and the browser can only handle one of those at once. If you use
gzip-compression on your web-server and download each file asynchronously, I
don''t think you''ll see much of a difference. However,
"minifying" Ruby or Python like most JavaScript is really not
possible, so people could always opt for a XAP file. But I really want to give
back the text-only experience, even for deployment. There are still good ways to
support asynchronous downloading of files that get imported (I''ll talk
about that below), so we should still explore those ... especially since this is
the way browsers work today.

Imagine if Ruby and Python in the browser felt not that much different than
JavaScript, but allow for other deployment options if the app needs it. For
example, a Python Silverlight Hello World app should just be this:

<!-- index.html -->
<script type="text/javascript"
src="silverlight-dlr.js"></script>
<div id="message"></div>
<script type="application/python">
document.message.innerHTML = "Hello, World!"
</script>

Very simple and clean. No need to start a web-server or install Mono on the Mac
just to make a client-side page. The JS file hosts Silverlight, and then
Microsoft.Scripting.Silverlight scans the HTML page looking for <script>
tags, executing them as they are encountered. Pre-packaged xap files are there
so a basic app would never need to touch the XAP. There is a project being
released today on http://visitmix.com/Lab which implements SL Ruby and Python
apps just like this, with the intention that the ideas about making things
simpler are merged back into Microsoft.Scripting.Silverlight. I''ll send
out links when it''s live, but hopefully you''ll just hear about
it. =)

FWIW, I''d leave the "entryPoint" activation in there as well,
so only if you didn''t set entryPoint OR didn''t have a valid
entry point in the XAP then we''d look at the HTML page for
<script> tags ... we can deprecate it if enough people stop using
entryPoint. Also, it''s a little strange to take a bunch of python
islands and assume it''s one module. Should those islands be their own
modules? Also, I know there might be some standards issues with doing a
<script type=""> when it''s not a list of approved
types. We''ll have to make sure this is done in a standards-compliant
way, as I''d love to see browsers support this out of the box (ha, yeah,
sounds crazy I know), and this work should be a stepping stone to that.
I''ve already reviewed the visitmix.com/Lab project and there are some
standards issues that I''ll blog about tomorrow, so please
don''t get caught up on a prototype or proposal, just imagine what
things could be like. =P

This scales to bringing in other files too, as you can use <script
type="application/python" src="foo.py" />.
Microsoft.Scripting.Silverlight would scan the HTML file for <script> tags
with the src attribute, and download the files ahead of time. But what happens
if those files "import" something? The visitmix project I was
referring to does not handle this case, since it''s just a prototype of
how browsers could support multiple languages. But we''d need to support
this. And this is where it could look back the XAP. So if we ever did take away
entryPoint then you''d activate your apps by just doing this:

<script type="application/python">
  import app
</script>

If an import failed against the XAP, it could look on the web-server. Though
today''s implementation is synchronous, that needs to change. So the
guidance would be to either to put those files in the XAP, or allow the
app-author to specify their dependencies up-front (like JavaScript does, it
could even be with <script> tags also) so those files could be downloaded
and import would just run the contents of the pre-downloaded files.
Unfortunately, even if we could do import asynchronously (by making
import/require/etc continuation-aware), the downloads would still happen one at
a time, which is unacceptable for deployment. So it''s either the
up-front-list or in the XAP.

WRT sharing IronPython assemblies between apps, that change was kinda silently
checked in around SL3 RTW since I wanted to test things a bit more, and find a
place to host the packages. Documentation will pop up on the IronPython codeplex
site in a few weeks about how this all works. And the way you host your apps
should still be caching it, but you''ll see a download request (in
Firebug for example). Are you sure they are being fully downloaded each time?
Maybe the AppManifest.xaml downloading is forcing a download each time, but I
remember it going to the browser cache.

I know that was kinda long winded, but I''m pretty happy about fixing a
lot of the ugliness we had to add from SL1.1 to SL2, while keeping the
deployment benefits SL2 had, even if they are in a different form.
Today''s model is too complicated for people coming from JavaScript to
get started, so lowering the bar to the <script> tag level may get the
attention of more developers. Let me know if anyone has more questions about
this.

~js

________________________________________
From: users-bounces at lists.ironpython.com [users-bounces at
lists.ironpython.com] on behalf of Michael Foord [fuzzyman at voidspace.org.uk]
Sent: Tuesday, July 21, 2009 2:37 AM
To: Discussion of IronPython
Cc: ironruby-core at rubyforge.org
Subject: Re: [IronPython] [Code Review] "XAP-less" Silverlight
application support

Hello Jimmy,

Will xap-less be an ''alternative'' deployment scenario for
dynamic
silverlight applications or the ''new and only'' deployment
scenario?

If it becomes the ''right'' way to do it please realise that you
are in
one swoop obsoleting all the articles, tutorials, examples (and books)
on developing Silverlight applications with IronPython / IronRuby. There
better be a big payoff for such a hugely backwards incompatible move...

When the move to xap files happened we were told this was a big win for
dynamic languages as all needed Python (or Ruby) files would be
downloaded compressed - and this was *much* faster than downloading them
synchronously on import (as the integration in Silverlight 1.1
originally did). What is the motivation for the change?

Does this change allow the IronPython assemblies to be shared between
applications (and cached)? I realise this can be done already (I use it
with my examples) - but I still see the IronPython assemblies being
re-downloaded every time you refresh the page or go to a new example,
even though they are from the same location.

Michael


Jimmy Schementi wrote:
> (Sending this review broadly as it shows a new direction DLR Silverlight
apps are taking)
>
>
http://github.com/jschementi/ironruby/commit/da6b54e226adfd3a18d8ad98d618c2350ebd8351
>
> Beginnings of "XAP-less" Silverlight application support. Note:
Silverlight apps are *required* to have a XAP file, but with this change it can
get down to just a container for the AppManifest.xaml file. Python/Ruby would
ship with a single XAP file pointing to the .slvx files for a given release.
>
> Adds a "downloadScripts" initParam to indicate whether paths to
script files and XAML files should be resolved to the web-server (relative to
the XAP), if the file is not found in the XAP. Also adds
"downloadScriptsFrom" to customize what path they are based off of.
Because of the opt-in nature of this change, existing DLR Silverlight apps
should not break.
>
> Files are downloaded synchronously as code runs, by having the
PlatformAdaptationLayer use a new BrowserVirtualFilesystem:
HttpVirtualFilesystem. So when a language''s include mechanism is used
(require , load, import, etc), it will download the file by using XmlHttpRequest
in synchronous mode. It will also cache each downloaded file''s
contents, so a file will only be downloaded once, even if it is included
multiple times. Note: future work will hopefully allow asynchronous includes, so
this can be production-ready. For now, it should only be used for development.
>
> Note: this change decreases the need for Chiron significantly. In this
mode, Chiron only creates a XAP file with the necessary assemblies and
AppManifest.xaml, if it doesn''t exist already. One could imagine
providing a XAP file for each language, and then Chiron wouldn''t be
needed for development. However, for using the latest language assemblies, and
auto-detection of languages used, it''s still useful for development. So
some changes have been made to Chiron to support this better:
>
> A "localApplicationRoot" appSetting has been added to Chiron to
adjust where it looks for script files (when determining what languages are
used). This allows you to make sure Chiron is looking in the same place your
application is downloading scripts or xaml from, so it can still generate a
correct XAP file for you. "application/ruby" and
"application/python" mime-types have been also added to Chiron also to
enable downloading of those script files.
>
> All of Chiron''s appSettings are now override-able as command-line
switches: /u[rlPrefix], /e[xtUrlPrefix], /l[ocalAppRoot]
>
> Run Merlin/Main/Hosts/SilverLight/Tests/tests/manual/test_foox/run.bat for
an example of how this works (it requires a Silverlight Debug build, so if you
run Merlin/Main/Languages/Ruby/Scripts/Dev.bat, just run "bsd" and
that will *b*uild *s*ilverlight in *d*ebug mode)
>
> Adds a stub for similar support through IsolatedStorage, but that will come
later.
>
> Also tweaks the DynamicEngine.CreateRuntimeSetup method to better support
other DLR-hosts in Silverlight
(http://lists.ironpython.com/pipermail/users-ironpython.com/2009-July/010780.html).
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

--
http://www.ironpythoninaction.com/

_______________________________________________
Users mailing list
Users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

To which project on visitmix do you refer? Or is it not yet published? I
didn''t see anything off-hand that looked like what you described.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/ironruby-core/attachments/20090721/08479dac/attachment.html>

It''s not online yet. I''ll let everyone know once it is.

~Jimmy
Sent from my phone

On Jul 21, 2009, at 7:38 AM, "Ryan Riley"
<ryan.riley at panesofglass.org> wrote:
> To which project on visitmix do you refer? Or is it not yet
> published? I didn''t see anything off-hand that looked like what
you
> described.
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core

"That project" is online now: http://visitmix.com/labs/gestalt/

And here''s my take:
http://blog.jimmy.schementi.com/2009/07/gestalt-ruby-and-python-in-browser.html
________________________________________
From: Jimmy Schementi
Sent: Tuesday, July 21, 2009 4:21 AM
To: Discussion of IronPython
Cc: ironruby-core at rubyforge.org
Subject: RE: [IronPython] [Code Review] "XAP-less" Silverlight
application support

Michael,

You just had to ask now =P ... I was going to blog about this more extensively
tomorrow, but here''s a preview of my thinking about this:

My motivation behind xap-less is to make DLR Silverlight apps less complex, and
more like how the web works. This change is just an experiment in that
direction, and I''m very open to any opinions folks have about this.

First things first though, this change make xap-less completely opt-in ... you
have to use the "downloadScripts=true" InitParams setting to enable
it. Without this param it will behave as it used to. Also, even with this
setting it will not download the file if it exists in the XAP. I was very
conservative about this change, as I''m just playing around with some
ideas.

Also, even if xap-less becomes the default, I''d make sure existing
deployments did not break; in the xap will always be an option for deployment
(and quite possibly the preferred way), but I want to get back to the place
where we didn''t need a XAP file and didn''t need Chiron for
development.

Eh, I know that was the messaging around XAP files being compressed and faster
for download times, but that''s because they were synchronously
downloaded and the browser can only handle one of those at once. If you use
gzip-compression on your web-server and download each file asynchronously, I
don''t think you''ll see much of a difference. However,
"minifying" Ruby or Python like most JavaScript is really not
possible, so people could always opt for a XAP file. But I really want to give
back the text-only experience, even for deployment. There are still good ways to
support asynchronous downloading of files that get imported (I''ll talk
about that below), so we should still explore those ... especially since this is
the way browsers work today.

Imagine if Ruby and Python in the browser felt not that much different than
JavaScript, but allow for other deployment options if the app needs it. For
example, a Python Silverlight Hello World app should just be this:

<!-- index.html -->
<script type="text/javascript"
src="silverlight-dlr.js"></script>
<div id="message"></div>
<script type="application/python">
document.message.innerHTML = "Hello, World!"
</script>

Very simple and clean. No need to start a web-server or install Mono on the Mac
just to make a client-side page. The JS file hosts Silverlight, and then
Microsoft.Scripting.Silverlight scans the HTML page looking for <script>
tags, executing them as they are encountered. Pre-packaged xap files are there
so a basic app would never need to touch the XAP. There is a project being
released today on http://visitmix.com/Lab which implements SL Ruby and Python
apps just like this, with the intention that the ideas about making things
simpler are merged back into Microsoft.Scripting.Silverlight. I''ll send
out links when it''s live, but hopefully you''ll just hear about
it. =)

FWIW, I''d leave the "entryPoint" activation in there as well,
so only if you didn''t set entryPoint OR didn''t have a valid
entry point in the XAP then we''d look at the HTML page for
<script> tags ... we can deprecate it if enough people stop using
entryPoint. Also, it''s a little strange to take a bunch of python
islands and assume it''s one module. Should those islands be their own
modules? Also, I know there might be some standards issues with doing a
<script type=""> when it''s not a list of approved
types. We''ll have to make sure this is done in a standards-compliant
way, as I''d love to see browsers support this out of the box (ha, yeah,
sounds crazy I know), and this work should be a stepping stone to that.
I''ve already reviewed the visitmix.com/Lab project and there are some
standards issues that I''ll blog about tomorrow, so please
don''t get caught up on a prototype or proposal, just imagine what
things could be like. =P

This scales to bringing in other files too, as you can use <script
type="application/python" src="foo.py" />.
Microsoft.Scripting.Silverlight would scan the HTML file for <script> tags
with the src attribute, and download the files ahead of time. But what happens
if those files "import" something? The visitmix project I was
referring to does not handle this case, since it''s just a prototype of
how browsers could support multiple languages. But we''d need to support
this. And this is where it could look back the XAP. So if we ever did take away
entryPoint then you''d activate your apps by just doing this:

<script type="application/python">
  import app
</script>

If an import failed against the XAP, it could look on the web-server. Though
today''s implementation is synchronous, that needs to change. So the
guidance would be to either to put those files in the XAP, or allow the
app-author to specify their dependencies up-front (like JavaScript does, it
could even be with <script> tags also) so those files could be downloaded
and import would just run the contents of the pre-downloaded files.
Unfortunately, even if we could do import asynchronously (by making
import/require/etc continuation-aware), the downloads would still happen one at
a time, which is unacceptable for deployment. So it''s either the
up-front-list or in the XAP.

WRT sharing IronPython assemblies between apps, that change was kinda silently
checked in around SL3 RTW since I wanted to test things a bit more, and find a
place to host the packages. Documentation will pop up on the IronPython codeplex
site in a few weeks about how this all works. And the way you host your apps
should still be caching it, but you''ll see a download request (in
Firebug for example). Are you sure they are being fully downloaded each time?
Maybe the AppManifest.xaml downloading is forcing a download each time, but I
remember it going to the browser cache.

I know that was kinda long winded, but I''m pretty happy about fixing a
lot of the ugliness we had to add from SL1.1 to SL2, while keeping the
deployment benefits SL2 had, even if they are in a different form.
Today''s model is too complicated for people coming from JavaScript to
get started, so lowering the bar to the <script> tag level may get the
attention of more developers. Let me know if anyone has more questions about
this.

~js

________________________________________
From: users-bounces at lists.ironpython.com [users-bounces at
lists.ironpython.com] on behalf of Michael Foord [fuzzyman at voidspace.org.uk]
Sent: Tuesday, July 21, 2009 2:37 AM
To: Discussion of IronPython
Cc: ironruby-core at rubyforge.org
Subject: Re: [IronPython] [Code Review] "XAP-less" Silverlight
application support

Hello Jimmy,

Will xap-less be an ''alternative'' deployment scenario for
dynamic
silverlight applications or the ''new and only'' deployment
scenario?

If it becomes the ''right'' way to do it please realise that you
are in
one swoop obsoleting all the articles, tutorials, examples (and books)
on developing Silverlight applications with IronPython / IronRuby. There
better be a big payoff for such a hugely backwards incompatible move...

When the move to xap files happened we were told this was a big win for
dynamic languages as all needed Python (or Ruby) files would be
downloaded compressed - and this was *much* faster than downloading them
synchronously on import (as the integration in Silverlight 1.1
originally did). What is the motivation for the change?

Does this change allow the IronPython assemblies to be shared between
applications (and cached)? I realise this can be done already (I use it
with my examples) - but I still see the IronPython assemblies being
re-downloaded every time you refresh the page or go to a new example,
even though they are from the same location.

Michael


Jimmy Schementi wrote:
> (Sending this review broadly as it shows a new direction DLR Silverlight
apps are taking)
>
>
http://github.com/jschementi/ironruby/commit/da6b54e226adfd3a18d8ad98d618c2350ebd8351
>
> Beginnings of "XAP-less" Silverlight application support. Note:
Silverlight apps are *required* to have a XAP file, but with this change it can
get down to just a container for the AppManifest.xaml file. Python/Ruby would
ship with a single XAP file pointing to the .slvx files for a given release.
>
> Adds a "downloadScripts" initParam to indicate whether paths to
script files and XAML files should be resolved to the web-server (relative to
the XAP), if the file is not found in the XAP. Also adds
"downloadScriptsFrom" to customize what path they are based off of.
Because of the opt-in nature of this change, existing DLR Silverlight apps
should not break.
>
> Files are downloaded synchronously as code runs, by having the
PlatformAdaptationLayer use a new BrowserVirtualFilesystem:
HttpVirtualFilesystem. So when a language''s include mechanism is used
(require , load, import, etc), it will download the file by using XmlHttpRequest
in synchronous mode. It will also cache each downloaded file''s
contents, so a file will only be downloaded once, even if it is included
multiple times. Note: future work will hopefully allow asynchronous includes, so
this can be production-ready. For now, it should only be used for development.
>
> Note: this change decreases the need for Chiron significantly. In this
mode, Chiron only creates a XAP file with the necessary assemblies and
AppManifest.xaml, if it doesn''t exist already. One could imagine
providing a XAP file for each language, and then Chiron wouldn''t be
needed for development. However, for using the latest language assemblies, and
auto-detection of languages used, it''s still useful for development. So
some changes have been made to Chiron to support this better:
>
> A "localApplicationRoot" appSetting has been added to Chiron to
adjust where it looks for script files (when determining what languages are
used). This allows you to make sure Chiron is looking in the same place your
application is downloading scripts or xaml from, so it can still generate a
correct XAP file for you. "application/ruby" and
"application/python" mime-types have been also added to Chiron also to
enable downloading of those script files.
>
> All of Chiron''s appSettings are now override-able as command-line
switches: /u[rlPrefix], /e[xtUrlPrefix], /l[ocalAppRoot]
>
> Run Merlin/Main/Hosts/SilverLight/Tests/tests/manual/test_foox/run.bat for
an example of how this works (it requires a Silverlight Debug build, so if you
run Merlin/Main/Languages/Ruby/Scripts/Dev.bat, just run "bsd" and
that will *b*uild *s*ilverlight in *d*ebug mode)
>
> Adds a stub for similar support through IsolatedStorage, but that will come
later.
>
> Also tweaks the DynamicEngine.CreateRuntimeSetup method to better support
other DLR-hosts in Silverlight
(http://lists.ironpython.com/pipermail/users-ironpython.com/2009-July/010780.html).
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

--
http://www.ironpythoninaction.com/

_______________________________________________
Users mailing list
Users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com