I have done a lot of thinking about the Application Database
http://appdb.codeweavers.com lately and how the rating system works. We
currently have 848 Applications in there but only 58 with a 5 star
rating which works out to less than 10% and thats the good news. The bad
news is that of those 58 only 26 rate that without windows installed.
After takeing a closer look, none of the apps I looked at actualy rated
5 star by my way of judging. Now maybe I'm overly critical but in my way
of thinking If the app doesn't install in wine then it shouldn't get a
5. IMHO We need to set criteria that give us a better/less subjective
way of measuring how well we are doing. So this is what I have come up with.
1. Installation (total possible 2)
Installs in wine with fake windows 2 points
Installs in wine with windows mounted 1 point
Does not install in wine 0 points
2. Running & DLLS (total possible 2)
Runs fine using all builtin dlls 2 points
Runs fine using native dlls 1 point
Does not run/crashes on startup 0 points
3 Running general (total possible 2)
program does not crash ever 2 points
Program crashes only after long use or rare aspect used 1 point
program crashes on startup or doing common function 0 points
4. Screen Presentation/layout (total possible 2)
App looks exactly like it should 2 points
Small problems that do not affect working with app 1 point
App is unusable because of display problems 0 points
5. Fonts (total possible 2)
Fonts look exacty right 2 points
Fonts not exactly right but acceptable 1 point
Fonts unreadable or totaly wrong. 0 points
6. Mouse (total possible 2)
Mouse behaves as expected 2 points
Mouse behaves strange but can still use app 1 point
Mouse behaves badly and is unusable 0 points
7. Keyboard (total possible 2)
Keyboard reacts as expected. 2 points
Some keys/key combinations dont work but most do. 1 point
Keyboard locks up or unresposive 0 points
8. Files save/restore (total possible 2)
Able to save and restore files without any problems 2 points
Can save and restore files but have problems 1 points
Cannot save or restore files 0 points
9. Sound (total possible 2)
Not applicable
Sound is correct 2 points
sound is ok most of the time but has minor glichs 1 point
Sound is garbled or missing 0 points
10.Joystick/gamepad (total possible 2)
Not applicable
joystick/gamepad works correctly 2 points
joystick/gamepad have problems but mostly work 1 point
joystick/gamepad unresponsive 0 points
11.Registy/rememberd state (total possible 2)
Not applicable
Program remembers key settings from one session to next 2 points
Program remembers most settings but not all 1 point
program does not remember most settings at all 0 points
12. Date tested. Verison of wine used
Each of the sections should have a comment field and a place to enter a
url to a bug report if applicable
Applications could be scored based as a percentage. Applications that
use all critera would need all 22 points to get a 100% rating. For other
applications some of these criteria are not applicable and their score
would use a lower base score.
Idealy we would be able to look at the AppDB and list all programs that
have problems with any of the criteria
Probably the most usefull aspect of this is that it gives users and
developers a real way of measuring how well wine really is doing and
where problems that affect the most programs are.
I have cross posted this because I think this is important to both users
and developers. Any comments, criticisims or helpfull suggestions welcome
Tony Lambregts
An excellent idea! --- Tony Lambregts <tony_lambregts@telusplanet.net> wrote:> > 12. Date tested. Verison of wine usedVersion of operating system? Hardware configuration?> For other applications some of these criteria arenot> applicable and their score would use a lower base > score.Would it be easier to just give the app the full points for that category? So "fully works or not-applicable" is two points, "sort of works" is one point, and "broke" is zero. ====Jon Miner http://www.ConcordNH.com/Trains - Concord Model RR Club http://bmrrhs.come.to - B&M RR Historical Society __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I Agree that this is a great idea. I always felt that the appdb left much to be desired. Kevin ====--------------------------------- Kevin DeKorte kdekorte@yahoo.com __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I also agree that this is a great idea and that the application database needs to be changed to more accurately reflect the state of wine. I'm still of the opinion that the applications matter the most. How can we get these changes implemented? How can we integrate our old data with this new format? We don't want to lose all the data we already have. I also agree that we need os/platform/window manager data and that n/a should be a 2, so we can one common score for complete applications. That will make the data much easier to understand. On Thu, May 02, 2002 at 09:53:15PM -0600, Tony Lambregts wrote:> I have done a lot of thinking about the Application Database > http://appdb.codeweavers.com lately and how the rating system works. We > currently have 848 Applications in there but only 58 with a 5 star > rating which works out to less than 10% and thats the good news. The bad > news is that of those 58 only 26 rate that without windows installed.-- michael cardenas lead windows compatibility engineer lindows.com "Be the change you want to see in the world" -Mahatma Gandhi
On Thu, May 02, 2002 at 09:53:15PM -0600, Tony Lambregts wrote:> I have done a lot of thinking about the Application Database > http://appdb.codeweavers.com lately and how the rating system works. We > currently have 848 Applications in there but only 58 with a 5 star > rating which works out to less than 10% and thats the good news. The bad > news is that of those 58 only 26 rate that without windows installed. > After takeing a closer look, none of the apps I looked at actualy rated > 5 star by my way of judging. Now maybe I'm overly critical but in my way > of thinking If the app doesn't install in wine then it shouldn't get a > 5. IMHO We need to set criteria that give us a better/less subjective > way of measuring how well we are doing. So this is what I have come up with. > > 1. Installation (total possible 2) > Installs in wine with fake windows 2 points > Installs in wine with windows mounted 1 point > Does not install in wine 0 points > > 2. Running & DLLS (total possible 2) > Runs fine using all builtin dlls 2 points > Runs fine using native dlls 1 point > Does not run/crashes on startup 0 points > > 3 Running general (total possible 2) > program does not crash ever 2 points > Program crashes only after long use or rare aspect used 1 point > program crashes on startup or doing common function 0 points > > 4. Screen Presentation/layout (total possible 2) > App looks exactly like it should 2 points > Small problems that do not affect working with app 1 point > App is unusable because of display problems 0 points > > 5. Fonts (total possible 2) > Fonts look exacty right 2 points > Fonts not exactly right but acceptable 1 point > Fonts unreadable or totaly wrong. 0 points > > 6. Mouse (total possible 2) > Mouse behaves as expected 2 points > Mouse behaves strange but can still use app 1 point > Mouse behaves badly and is unusable 0 points > > 7. Keyboard (total possible 2) > Keyboard reacts as expected. 2 points > Some keys/key combinations dont work but most do. 1 point > Keyboard locks up or unresposive 0 points > > 8. Files save/restore (total possible 2) > Able to save and restore files without any problems 2 points > Can save and restore files but have problems 1 points > Cannot save or restore files 0 points > > 9. Sound (total possible 2) > Not applicable > Sound is correct 2 points > sound is ok most of the time but has minor glichs 1 point > Sound is garbled or missing 0 points > > 10.Joystick/gamepad (total possible 2) > Not applicable > joystick/gamepad works correctly 2 points > joystick/gamepad have problems but mostly work 1 point > joystick/gamepad unresponsive 0 points > > 11.Registy/rememberd state (total possible 2) > Not applicable > Program remembers key settings from one session to next 2 points > Program remembers most settings but not all 1 point > program does not remember most settings at all 0 points > > 12. Date tested. Verison of wine used >Way too complicated IMHO. Do you really think people will put up with this system if currently they very frequently don't even mention whether they actually tried the app on Wine and what results they got ? (yes, I added a comment which asks them to also mention this) A lot of the submissions that I commit leave several things to be desired, too (which I frequently improve a lot before committing, which shouldn't really be my job, though). IMHO the fact that the current *easy* rating system doesn't get used properly sort of indicates that it would be even worse with a more complicated system. One possible bad result: less people putting up with submitting apps, because it's "too much work", thus dramatically fewer apps listed. Do we want that ? IMHO we want as many apps listed as possible, even with a somewhat fuzzy picture, as opposed to simply listing Office, IE, HalfLife and Starcraft with perfect rating and description. People do want to know whether a particular highly weird app (take "Femta" as an example ;-) might have the slightest chance of running on Wine, so having a relatively broad base is also pretty important. -- Andreas Mohr Stauferstr. 6, D-71272 Renningen, Germany Tel. +49 7159 800604 http://home.arcor.de/andi.mohr/
On Thu, 2 May 2002, Tony Lambregts wrote:> So this is what I have come up with. > > 1. Installation (total possible 2)[...]> 12. Date tested. Verison of wine usedWith the proviso that having multiple categories tends to favour "average-looking" results, the system looks good. I would also include the license under which the software is available. Without wishing to reigniting the wine-license debate :) the license could be one of: "free w/ sourcecode", "free but binary only", "commercial", "Donno, got it off a mate".> Idealy we would be able to look at the AppDB and list all programs that > have problems with any of the criteriaYes. Also, it would be nice if you could search the db for applications that have a certain range of points (or stars), rather than just >= x, for example to look for all four-star applications.> Probably the most usefull aspect of this is that it gives users and > developers a real way of measuring how well wine really is doing and > where problems that affect the most programs are.Hmm, there was always the possibility of this, but I think any half-way reliable estimate would require retesting working apps; or do you assume no regression in functionality? What do you think? ---- Paul Millar