weft qda users - Oct 2007 - [Weft QDA users] Using WeftQDA for Mailing-List Analysis

Hi Alex, Hi Weft-Users!
   I am looking for a tool for doing a qualitative analysis of mailing-list  
data. This means I have a *lot* of individual documents (the typical case:  
around 500-5000 emails), belonging to the same "super-document" (a
mailing
list).

I have looked at several tools and I like that Weft is public domain and  
written in Ruby, so I want to figure out whether it suits my set of  
requirements.

In particular I would like to find out, whether there is a possibility  
that I could extend WeftQDA to allow me to import a mailbox-file as a  
hierarchical set of documents. Idealy I would like these documents to be  
sorted by date with replies being nested documents with some ability to  
search by author and subject of the email.

One could put this as a more general feature for importing and working  
with structured data. For instance if one has conducted several interviews  
at different sites, the sites would make a good meta-data attribute to  
answer questions like: Is there a difference between responses at  
different sites? Are certain categories predominantly clustered at for  
instance one company? Also dates as metadata might be useful: Did the  
answers change over time as the subject was interviewed repeatedly.

Would this be of interest to anybody?

Which branch/tag/trunk should I checkout if I want to have a look at the  
codebase?

Thank you,
   Christopher

-- 
Christopher Oezbek
Arbeitsgruppe Software Engineering
Institut f?r Informatik
Freie Universit?t Berlin
Takustr. 9, 14195 Berlin, Germany
+49 30 838 75242, Raum 008
http://www.inf.fu-berlin.de/~oezbek/

Hi Christopher

Christopher Oezbek wrote:
> I have looked at several tools and I like that Weft is public domain and  
> written in Ruby, so I want to figure out whether it suits my set of  
> requirements.
>   
Thanks for your interest.
> In particular I would like to find out, whether there is a possibility  
> that I could extend WeftQDA to allow me to import a mailbox-file as a  
> hierarchical set of documents. Idealy I would like these documents to be  
> sorted by date with replies being nested documents with some ability to  
> search by author and subject of the email.
>   
I''d say that the current release would not really meet those 
requirements that well, and you might fare better with one of the 
commercial tools. It doesn''t really have a notion of hierarchical 
documents. It does have a fairly fast free-text search, but it is also 
quite expensive in terms of disk space. If you wanted to import a lot of 
documents and create automatic Categories on them, you''d probably need 
to write a bit of Ruby or a bit of SQL.
> One could put this as a more general feature for importing and working  
> with structured data. For instance if one has conducted several interviews
> at different sites, the sites would make a good meta-data attribute to  
> answer questions like: Is there a difference between responses at  
> different sites? Are certain categories predominantly clustered at for  
> instance one company? Also dates as metadata might be useful: Did the  
> answers change over time as the subject was interviewed repeatedly.
>   
Absolutely, this is an important feature. Weft current can only fake 
this by using Categories to code whole Documents. This, in conjunction 
with the Query and CodeReview tools works reasonably well, though not 
perfectly. For example, you can use the CodeReview tool to get a quick 
comparison of the number of passages from SiteX and SiteY that are coded 
with a certain (analytic) category.

The development version of Weft has proper "Attributes"; you can
define
any number of attributes (eg sex) and sets of values that those can take 
(male/female), and apply them and view documents by them. Unfortunately 
other commitments have meant that Weft 2 hasn''t emerged yet, but
it''s
still underway.
> Which branch/tag/trunk should I checkout if I want to have a look at the  
> codebase?
>   
The latest Weft 1 stable code is on this branch, but is very dated
http://weft-qda.rubyforge.org/svn/branches/stable_1-0/

The latest development code is here:
http://weft-qda.rubyforge.org/svn/trunk/weft-qda/

In trunk, most of the important things are working: importing, coding, 
viewing, applying attributes, searching, saving, exporting etc. The code 
is also a lot easier to follow. What''s missing is updated versions of 
the Query and CodeReview tools that handle attributes.

cheers
alex

Hi Alex,
   thanks for the fast reply!
> The latest development code is here:
> http://weft-qda.rubyforge.org/svn/trunk/weft-qda/
>
> In trunk, most of the important things are working: importing, coding,
> viewing, applying attributes, searching, saving, exporting etc. The code
> is also a lot easier to follow. What''s missing is updated versions
of
> the Query and CodeReview tools that handle attributes.
I tried to run it from SVN, but I got following error:

weft-qda/lib/weft/wxgui/dialogs/category_properties.rb:2: uninitialized  
constant QDA::GUI::CategoryPropertiesDialogBase (NameError)

I fixed that using the patch shown below.

Which seems to make it run (I have not looked much further).

Can you recommend any development environment for working with the weft  
code base? I am particularily overwhelmed by the many modules and their  
classes in different files and directories.

Thanks,
   Christopher

Index: D:/workspaces/ruby/weft-qda/lib/weft/wxgui/base.rb
==================================================================---
D:/workspaces/ruby/weft-qda/lib/weft/wxgui/base.rb	(revision 1008)
+++ D:/workspaces/ruby/weft-qda/lib/weft/wxgui/base.rb	(working copy)
@@ -35,7 +35,34 @@
    end
  end

+class QDA::GUI::CategoryPropertiesDialogBase < Wx::Dialog
+
+  attr_reader :title_box, :create_date, :mod_date, :memo_box
+
+  def initialize(parent = nil)
+    super()
+    xml = Wx::XmlResource.get
+    xml.flags = 2 # Wx::XRC_NO_SUBCLASSING
+    xml.init_all_handlers
+    xml.load("xrc/weft-qda.xrc")
+    xml.load_dialog_subclass(self, parent,
"CategoryPropertiesDialog")

+    finder = lambda do | x |
+      int_id = Wx::xrcid(x)
+      begin
+        Wx::Window.find_window_by_id(int_id, self) || int_id
+      # Temporary hack to work around regression in 1.9.2
+      rescue RuntimeError
+        int_id
+      end
+    end
+
+    @title_box = finder.call("title_box")
+    @create_date = finder.call("create_date")
+    @mod_date = finder.call("mod_date")
+    @memo_box = finder.call("memo_box")
+  end
+end

  # This class was automatically generated from XRC source. It is not
  # recommended that this file is edited directly; instead, inherit from



-- 
Christopher Oezbek
Arbeitsgruppe Software Engineering
Institut f?r Informatik
Freie Universit?t Berlin
Takustr. 9, 14195 Berlin, Germany
+49 30 838 75242, Raum 008
http://www.inf.fu-berlin.de/~oezbek/

[I think this is probably getting too technical and OT for this list, so 
please follow-up in email - thanks. If you''re only interested in using 
Weft QDA, skip this message.]

Christopher Oezbek wrote:
> I tried to run it from SVN, but I got following error:
>
> weft-qda/lib/weft/wxgui/dialogs/category_properties.rb:2: uninitialized  
> constant QDA::GUI::CategoryPropertiesDialogBase (NameError)
>   
The patch looks good, although that''s an auto-generated file created by
running "rake rbxrc" on the command line.

I''ve not needed to be particularly careful about keeping the SVN
version
working but if others are looking at the code I''ll be more
attentive.
> Can you recommend any development environment for working with the weft  
> code base?
I''m using Emacs, which is fine if you''re familiar with it.
I''ve also
found Eclipse + Ruby Development Tools works well for a large project 
like this, because it has a debugger, test unit harness and 
click-to-jump to exception backtrace.
>  I am particularily overwhelmed by the many modules and their  
> classes in different files and directories.
>   
Sure, it''s a pretty large codebase. Luca Manini and co at the
University
of Trento were looking at it, and I drew up an architecture diagram 
which I lost when my laptop got stolen. So if any Trento people have 
that, please could you send me a copy?

As an overview of what''s where:

* lib/ contains all the ruby code for Weft QDA
* share/ contains icons, the SQL schema, and templates for export
* test/ is a reasonable test suite
* rake/ are tasks for creating .exes, Windows installers
* xrc/ has XML layout descriptions of GUI windows

The non-GUI code is separate from the desktop application code:

* lib/weft/ contains the code for the core classes of the application, 
like Document, Category, Search, Code, Query, Fragment
* lib/weft/application.rb is the central class for setting up a database
* lib/weft/backend/sqlite is where the code that handles the SQLite 
database as an application file store is: opening, closing, saving, 
autosaving etc
* lib/weft/filters contains code for importing and exporting to external 
files

The GUI code is the largest part:
* lib/weft/wxgui/client.rb is the main application code for opening and 
closing project files etc
* lib/weft/wxgui/inspectors is where the code for the individual windows 
for viewing and marking Documents, Categories etc lives
* lib/weft/wxgui/controls is where the important GUI controls for 
picking, listing, editing and viewing application objects
* lib/weft/wxgui/sidebar contains the code for the main application 
window contents

Architecturally, the desktop application has an instance of WeftClient 
connected to a QDA::Application. Once an application is started (loaded 
from a file), the GUI classes load application objects by calling, for 
example, Document.find or Document.find_all. 

User actions in the GUI mean that changes are applied to objects using, 
for example, Document.save. This triggers a message to all interested 
GUI objects that the object has changed, and they then update their 
state to reflect the new object (eg, changed text in a Document).

That''s very brief, drop me a line off-list if you''ve got more
questions:

cheers
alex