compiz - Dec 2008 - [ANNOUNCE] Compiz feature branch compiz++

Hi,

I've currently pushed a new branch called "compiz++" to the
freedesktop
repository, with some features I've been working on during last months. 
Because most of the features also require (BIG) changes to the plugins, I've
decided to put them all together.

- No direct access to member variables: Everything is now done with getter and 
setter functions. This helps with the problem where every variable addition 
broke the plugin ABI. While this usually is not a problem in the development 
tree, we can not do it in a stable tree.  Even if it is needed to fix a bug. 
It also gives us more control what other plugins can do, so that broken 
plugins can not mess up our core structures.

- Composite/OpenGL seperation: All XComposite handling is moved into the 
composite plugin and the opengl rendering into the opengl plugin. This allows 
compiz to run also as normal window manager without compositing (still needs 
some work and a few new plugins), but also the creation of other rendering 
backends (XRender, clutter, ...).

- Tiled textures: Modifications to the texture system allow to have more than 
one texture per pixmap and also to have other plugins that provide texture 
from pixmap functionality. The new copytex uses the (slow) "copy pixmap 
content to texture" approach to provide texture from pixmap functionality. 
It's main advantage is that it supports pixmaps bigger than the maximum 
texture size. In this cases a pixmap is split into multiple textures. Compiz 
will use the glx texture-from-pixmap extension for pixmaps/windows but will 
fall back to the copytex plugin if thex are bigger than the maximum texture 
size. 

- Reparented decorations: Like other window managers, compiz now supports 
reparented window decorations. This will allow compiz to run as a normal 
window manager and to be able to have decorations for windows that are bigger 
than the maximum texture size (copytex plugin). (Currently only implemented in 
the kde4-window-decorator)

- Dropped multi display and multi screen support *: The multi display support 
is not completed, and the multi screen support is almost unmaintainded. 
Additionally, our normal proposal for bugs in the multi screen support is to 
start one compiz instance per screen. Dropping multi-screen support has the 
additional benifits of:
-- Per screen plugin lists (e.g. cube on one screen and wall on the other)
-- Rendering of one screen can not block the rendering of the other screen 
-- Different libGL per screen (with LD_PRELOAD)
-- Simplier plugins 
-- Simplier option handling

- New plugin interface: The compiz WRAP/UNWRAP interface is perhaps the most 
efficient system to create a plugin funcion "call chain", but it is
not the
the best for compiz. In compiz we have a lot of plugins loaded that hook into 
the drawing functions, but do nothing most of the time. Only when activated 
(keybinding/dbus/event) do they usually draw something. The end result is that 
most functions get called, then the plugin checks if it is active. The plugin 
then calls the function of the next plugin in the call chain.  The following 
plugin repeats the cycle. I've measured that we are loosing 8 - 15 % of CPU 
time here. The new system allows plugins to dynamically disable
"wrapped"
functions if they are not needed. The core then only needs to check some 
boolean variables, and will only call the functions that are really in use. 
I've kept the word "wrapping" in the system so that every compiz
developer
knows what it does, even if it does not fit the new system well.

- CMake build system: Everyone is happy if the build system (autotools) works, 
but starts to cry if it doesn't. At least that is my impression after more 
than 2 years of compiz development. In compiz fusion we have already decided 
to switch to cmake after the next stable release (0.8), and we also provided 
cmake tarballs for a while. CMake provides a real great documentation, and in 
my opinion it is much easier than autotools.

- Port to C++: With C we have to write almost the same code in each plugin and 
run into the same bugs over and over again. A lot of this can be avoided with 
C++. C++ allows us also to do more and this in a easier way:
-- Smarter function callbacks with Boost (boost::function/boost::bind)
-- Easier and smarter privates system (get the plugin struct for a given core 
struct/old FOO_SCREEN (s), FOO_WINDOW (w) macros). The new system hides most 
of the ugly handling from a plugin developer and provides a new simple and ABI 
safe way to work with plugin plugins (plugins that expose 
features/functionality to other plugins)
-- Constructors/Destructors allow easier initialisation/cleanup for a lot of 
systems.
-- Containers avoid the implementation of lists and resizeable arrays over and 
over again. 
-- Containers like maps and other smart classes can improve performance in 
several areas.
-- ...

This branch is my proposal for a possible compiz future. The decision to make 
this the future of compiz is something that everyone involved in compiz should 
make. Espesially because it would need a huge amount of work to get all 
plugins ported. In my opinion, it would be better to stay with the old system 
if such change would create a fork again.

I've also pushed compiz++ branches into the libcompizconfig, bcop and 
crashhandler repositories. They contain some initial work for the compiz++ 
branch. The crashhandler branch also contains a modified (not finished) 
version of the universal Makefile.

Merry Christmas
Dennis Kasprzyk

* multi display: Connect to multiple x servers 
  multi screen: seperated screens (usually different graphic cards) not 
connected with xinerama
  multi head: multiple monitors in a xinerama/randr 1.2 configuration (still 
supported)

'Twas brillig, and Dennis Kasprzyk at 24/12/08 12:48 did gyre and gimble:
<snip>

Just a quick query, is this an all-or-nothing proposal or can some bits 
be split out? e.g. the C++ bit... I'd imagine it's the most 
controversial part of the proposal.

I'm in no position to comment in any sort of educated way about C vs. 
C++ in compiz, but I'd generally suspect that C would still be the 
preferred route by most developers even if it is more effort.

Col

-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
   Tribalogic Limited [http://www.tribalogic.net/]
Open Source:
   Mandriva Linux Contributor [http://www.mandriva.com/]
   PulseAudio Hacker [http://www.pulseaudio.org/]
   Trac Hacker [http://trac.edgewall.org/]

On Wed, Dec 24, 2008 at 01:48:17PM +0100, Dennis Kasprzyk
wrote:
> - No direct access to member variables: Everything is now done with
> getter and setter functions. This helps with the problem where every
> variable addition broke the plugin ABI. While this usually is not a
> problem in the development tree, we can not do it in a stable tree.  Even
> if it is needed to fix a bug.  It also gives us more control what other
> plugins can do, so that broken plugins can not mess up our core
> structures.
How does this relate to the fundamental issue where we simply have way too
many variables available under one or two data structures? I believe the
biggest problem we've had is the sheer size of the core structures and their
lack of documentation which has led to misuse. The get/set for ABI
compatibility is not unimportant, but it doesn't solve the underlying
problem.
> - Composite/OpenGL seperation: All XComposite handling is moved into the 
> composite plugin and the opengl rendering into the opengl plugin. This
allows
> compiz to run also as normal window manager without compositing (still
needs
> some work and a few new plugins), but also the creation of other rendering 
> backends (XRender, clutter, ...).
What are the specifics for this interface?
> - Tiled textures: Modifications to the texture system allow to have more
than
> one texture per pixmap and also to have other plugins that provide texture 
> from pixmap functionality. The new copytex uses the (slow) "copy
pixmap
> content to texture" approach to provide texture from pixmap
functionality.
> It's main advantage is that it supports pixmaps bigger than the maximum
> texture size. In this cases a pixmap is split into multiple textures.
Compiz
> will use the glx texture-from-pixmap extension for pixmaps/windows but will
> fall back to the copytex plugin if thex are bigger than the maximum texture
> size. 
What are the specifics for this interface?
> - Reparented decorations: Like other window managers, compiz now supports 
> reparented window decorations. This will allow compiz to run as a normal 
> window manager and to be able to have decorations for windows that are
bigger
> than the maximum texture size (copytex plugin). (Currently only implemented
in
> the kde4-window-decorator)
What specific bugs does this solve, what are the concerns that it will
bring? What is the reason for the original design? Pros and cons.

Is this an optional feature, or is it always enabled in your branch? Is it
implemented as a plugin?
> - Dropped multi display and multi screen support *: The multi display
support
> is not completed, and the multi screen support is almost unmaintainded. 
> Additionally, our normal proposal for bugs in the multi screen support is
to
> start one compiz instance per screen. Dropping multi-screen support has the
> additional benifits of:
> -- Per screen plugin lists (e.g. cube on one screen and wall on the other)
> -- Rendering of one screen can not block the rendering of the other screen 
> -- Different libGL per screen (with LD_PRELOAD)
> -- Simplier plugins 
> -- Simplier option handling
I believe this has been a separate discussion, though it seems it never
came up on any of the lists(?). I believe a proper discussion should be had (I
could've sworn we had one already), but I suspect this could be implemented
on master without too much grief.
> - New plugin interface: The compiz WRAP/UNWRAP interface is perhaps the
most
> efficient system to create a plugin funcion "call chain", but it
is not the
> the best for compiz. In compiz we have a lot of plugins loaded that hook
into
> the drawing functions, but do nothing most of the time. Only when activated
> (keybinding/dbus/event) do they usually draw something. The end result is
that
> most functions get called, then the plugin checks if it is active. The
plugin
> then calls the function of the next plugin in the call chain.  The
following
> plugin repeats the cycle. I've measured that we are loosing 8 - 15 % of
CPU
> time here. The new system allows plugins to dynamically disable
"wrapped"
> functions if they are not needed. The core then only needs to check some 
> boolean variables, and will only call the functions that are really in use.
> I've kept the word "wrapping" in the system so that every
compiz developer
> knows what it does, even if it does not fit the new system well.
How exactly does this new system work? This represents a major change for
Compiz, and far more important than performance is the maintainability over
time. How do you envision this new system handling in the development
cycle? 

When are these 8-15% of CPU time? Is it during a time when Compiz is
heavily into damaging (ie: during animation) or a similarly intense
situation, thus the 8-15% of CPU time is actually 8-15% of the CPU, or is
it measured when Compiz is otherwise idle, and the 8-15% is 8-15% of ~0?

Generally speaking, my Compiz uses roughly no CPU, and if it's 8-15% of
nothing, I hardly think it's worth the gain, which is why this matters. But
depending on the implementation, this can probably be done pretty cleanly
without too much tinkering with existing plugins.
> - CMake build system: Everyone is happy if the build system (autotools)
works,
> but starts to cry if it doesn't. At least that is my impression after
more
> than 2 years of compiz development. In compiz fusion we have already
decided
> to switch to cmake after the next stable release (0.8), and we also
provided
> cmake tarballs for a while. CMake provides a real great documentation, and
in
> my opinion it is much easier than autotools.
This probably should be done directly on master after 0.8, but I'm still
missing some specifics. So far, all I've heard about CMake have been
praise, and I find it hard to believe entirely in a product when all I hear
is good stuff. There's always a con... Does anyone care to enlighten me and
the list?
> - Port to C++: With C we have to write almost the same code in each plugin
and
> run into the same bugs over and over again. A lot of this can be avoided
with
> C++. C++ allows us also to do more and this in a easier way:
> -- Smarter function callbacks with Boost (boost::function/boost::bind)
> -- Easier and smarter privates system (get the plugin struct for a given
core
> struct/old FOO_SCREEN (s), FOO_WINDOW (w) macros). The new system hides
most
> of the ugly handling from a plugin developer and provides a new simple and
ABI
> safe way to work with plugin plugins (plugins that expose 
> features/functionality to other plugins)
> -- Constructors/Destructors allow easier initialisation/cleanup for a lot
of
> systems.
> -- Containers avoid the implementation of lists and resizeable arrays over
and
> over again. 
> -- Containers like maps and other smart classes can improve performance in 
> several areas.
> -- ...
I'm sceptical to switching to C++ _and_ implementing all these changes at
the same time. Specially the way it's happened. I'm not entirely sure
C++
will solve all the real problems either (like the biggest problem which is
lack of good documentation). For now, I'd like to discuss the individual
changes suggested, or presented, I should say. Before I'm willing to
discuss the move to C++. 
> This branch is my proposal for a possible compiz future. The decision to
make
> this the future of compiz is something that everyone involved in compiz
should
> make. Espesially because it would need a huge amount of work to get all 
> plugins ported. In my opinion, it would be better to stay with the old
system
> if such change would create a fork again.
I'm really alarmed with how this branch/project evolved, so I'm
sceptical.
Essentially 6 months of development has been poured into a project we're
not supposed to talk about that changes more or less everything that is
compiz, and there's been no open discussion along the way. This has been an
ongoing problem with Compiz for as long as I've been involved, and it has,
in fact, been the single biggest challenge in working with Compiz. I regret
to see that you are carrying this tradition on Dennis...

I am in no position to reject or accept such a huge proposal, but I'm very
curious as to how to maintain it. Essentially we're turning everything into
a 0.2 version again, since we'll have to learn Compiz all over again.

I know I might have uttered differently before, but I've come to realize
that you do not solve the fundamental problems of a program just by
starting from scratch, and this is close to that.

From my point of view, it is much better to present each of these changes
as individual ideas, work over the idea, then start looking at an
implementation and patches. I see a lot of good ideas, but I don't see any
discussion with regards to the design or implementation. This will require
much work in the area of documentation and communication before I, at
least, am willing to give this a thumbs up (to the degree my thumb matters
in this situation).

For the record. I still agree with many of the problems you put forward,
but I believe that the biggest challenge in Compiz for the future lays not
in the code, but in the way we work, and in changing compiz from a research
project to a throughly professional project. That, and the lack of time.

- Kristian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url :
http://lists.freedesktop.org/archives/compiz/attachments/20081225/678da97c/attachment.pgp

On Wed, Dec 24, 2008 at 01:48:17PM +0100, Dennis Kasprzyk
wrote:
> Hi,
> 
> I've currently pushed a new branch called "compiz++" to the
freedesktop
> repository, with some features I've been working on during last months.
> Because most of the features also require (BIG) changes to the plugins,
I've
> decided to put them all together.
I've thought long and hard about Compiz++ and Compiz. This is a really hard
choice. I've talked around to try to get the general consensus, and I
believe the essence of what I'm about to say mirror the feelings of several
other developers. Since I can only speak for myself, I'll write
"I", though
in many cases, it's not just my own thoughts that's gone into this.

Essentially, I want to go ahead and develop Compiz based on master, not
Compiz++, but I can't deny that many of the features you've added in
Compiz++ should be implemented in Compiz. I believe that a switch to C++
was premature at best, and looking at core, I believe it will put us back
years with regards to a stable release.

What I want to do, is release Compiz 0.8.0, then focus on getting Compiz
1.0 out the door. This should be a matter of cleanups, documentation and
generally boring but necessary work. The goal should be to maintain Compiz
1.x as a binary compatible branch that is easy and comfortable to work
with both for Compiz developers, plugin developers and package maintainers. 

When Compiz 1.0 is out, we can start doing what you've already done,
except I still want to do it in C, because I do not believe C++ is the
answer to all our problems. In other words, I want to move 3D rendering
into a plugin, I want to investigate reparenting and tiled textures, and so
on.

Some of the features that you've introduced could probably go into 0.9
and thereby end up in 1.0. Removal of multi-screen support _could_ be one
of those features, as could the WRAP/UNWRAP replacement. Neither of these
features really require C++, and both have very clear advantages without
introducing too many new potential bugs.

The main reason that I do not want to go ahead with Compiz++, is that it's
simply too much in one bite. I'm really saddened by the way Compiz++
evolved. If it had come at an other time, and been developed openly and in
a community that had an open minded and interactive project management, 
I have no doubt that the end result would've been different. I also
understand that much of the reason that it was kept secret was to avoid the
risk of being overwritten by other branches (of and nomad), and that's a
sad testament to the state of Compiz as a community.

My only hope now is that Compiz can move forward, and that we can gain new,
active developers without loosing any old ones. My biggest fear is that you
(Dennis), will detach yourself from the project, which would be a huge
setback for Compiz, as your ideas, your understanding of Compiz and the
environment around it and your friendship has meant much both to Compiz and
to me personally. 

And I do not believe that Compiz++ was done in vain even if it is left
unmerged. The experiences gained in it's development can benefit Compiz, if
you let it.

- Kristian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url :
http://lists.freedesktop.org/archives/compiz/attachments/20090105/db6a8b82/attachment.pgp