Article: Usability of Open Source Software

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Article: Usability of Open Source Software

Andreas Bartel
Hello members,

here is a link to a well elaborated article about usability in open
source software.

http://www.cs.waikato.ac.nz/~daven/docs/oss-wp.html

The abstract:

Open source communities have successfully developed many pieces of
software although most computer users only use proprietary applications.
The usability of open source software is often regarded as one reason
for this limited distribution. In this paper we review the existing
evidence of the usability of open source software and discuss how the
characteristics of open-source development influence usability. We
describe how existing human-computer interaction techniques can be used
to leverage distributed networked communities, of developers and users,
to address issues of usability.

I encourage everyone to read the article. Many things that are described
there are 1:1 applicable to our situation, including positive as well as
negative observations. Maybe we can start a fruitful discussion based on
the suggestions made in the article about how to structure UX
participation in order to improve our product in terms of user needs and
usability.

Whether or not you're interested in UX matters, have a look, general
community work/life is also discussed.

Best,
Andreas

--
Sun Microsystems GmbH                Andreas Bartel
Nagelsweg 55                         User Experience Engineer
20097 Hamburg                        Phone: (+49 40)23646 672
Germany                              Fax:   (+49 40)23646 550
http://www.sun.com                   mailto:[hidden email]

Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028
Geschäftsführer: Thomas Schröder, Wolfgang Engels, Dr. Roland Bömer
Vorsitzender des Aufsichtsrates: Martin Häring


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Clément Pillias
Hi Andreas,

Le 3 févr. 09 à 16:43, Andreas Bartel a écrit :

> here is a link to a well elaborated article about usability in open  
> source software.

Unfortunately, the article is seven year old! Things have changed  
since then.

Moreover, there are a lot of unproven assumptions. I particularly  
disagree with the section titled "Usability problems are harder to  
specify and distribute than functionality problems". I do it each  
time I discuss an issue here. And being a former programmer, I know  
that code has the same kind of issues than UI concerning "design  
inconsistency" and "major overhaul". But maybe I am an exception  
without knowing it, and breaking issues into independent parts is not  
a common task for UX specialists?

The section "Open source projects lack the resources to undertake  
high quality usability work" assume that the only way to make "high  
quality usability work" is the way developed by and for closed-source  
software developers. I am opposed to that vision, and I think that,  
being open source, we have far more resources than closed-source  
software. But we still don't know how to use it efficiently :(

The section "OSS has an even greater tendency towards software bloat  
than commercial software" is simply wrong. Although the effects  
presented in this section are real, there are counter-effects, such  
as the need to maintain and reuse code, or the peer pressure to make  
the code evolve (instead of adding another piece of code). And the  
paragraph about early adopters could apply as well to closed-source  
software.

The section "Involving users" makes me think about IssueTracker  
statistics: if most users do not submit issues, can we say that most  
issues are submitted by developers, or is there a "long tail" effect  
and most issues are submitted by occasional submitters?

Best Regards,

Clément.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Andreas Bartel
Hi Clément,

very happy that you actually read the paper. Here are my two cents:

Clément Pillias schrieb:
> Hi Andreas,
>
> Le 3 févr. 09 à 16:43, Andreas Bartel a écrit :
>
>> here is a link to a well elaborated article about usability in open
>> source software.
>
> Unfortunately, the article is seven year old! Things have changed
> since then.
I don't thing that time is an issue. I'd really like to know what
specifically has changed and to what extent.
>
> Moreover, there are a lot of unproven assumptions. I particularly
> disagree with the section titled "Usability problems are harder to
> specify and distribute than functionality problems". I do it each time
> I discuss an issue here. And being a former programmer, I know that
> code has the same kind of issues than UI concerning "design
> inconsistency" and "major overhaul". But maybe I am an exception
> without knowing it, and breaking issues into independent parts is not
> a common task for UX specialists?
The first sentence of this section "Functional problems are easier to
specify, evaluate and modularize than certain usability problems." That
said, I totally agree with the title of this section. If you would
really dig deep and start thinking about the target user group, their
tasks and their context in order to specify a feature and how is should
behave, the whole process becomes much more complicated than creating a
functional specification when the requirements are known. I myself did
also a lot of programming in Java, C/C++ and Matlab in various
distributed projects before I started at Sun. In contrast to a quick and
dirty UX solution, a piece of quick and dirty code does not have to
cause confusion by the user or a whole user group.

The second important point that the authors show in this section is the
involvement of many designers when they work autonomously and thereby
generate design inconsistencies. Not on purpose of course. That
assumption is supported by a journal paper (sorry, forgot the title)
that pointed out that even UX professionals that evaluate an interface
independently, very rarely find the same usability bugs or generate
similar solutions. Meaning, the overlap of uncovered and solved UX
problems is very small in expert reviews.

Back to your question. Devide and conqure would not always work in UX.
Image you are designing a new way how to improve table handling in OOo.
You find a pritty cool solution for that rather "small" problem.
However, a user might like and learn the way you designer table handling
in Writer and would expect that tables in Impress, Base and even Calc
can be handled the same way. Or worse, a user might expect that handling
of other objects e.g. images, media etc. can be done in an at least
similar way. That would require to adapt your solution to all the cases
which suite user expectations. The handling of "Insert" table rows and
columns in Writer and Calc is a good example for inconsistancy.
>
> The section "Open source projects lack the resources to undertake high
> quality usability work" assume that the only way to make "high quality
> usability work" is the way developed by and for closed-source software
> developers. I am opposed to that vision, and I think that, being open
> source, we have far more resources than closed-source software. But we
> still don't know how to use it efficiently :(
Oooh, I very much see that point differently and I really would like
read what you have here in mind. UX methods are actually independent of
CSS or OSS, I think :-) However, first I would really really ask you to
give details here. Many thanks!
>
> The section "OSS has an even greater tendency towards software bloat
> than commercial software" is simply wrong. Although the effects
> presented in this section are real, there are counter-effects, such as
> the need to maintain and reuse code, or the peer pressure to make the
> code evolve (instead of adding another piece of code). And the
> paragraph about early adopters could apply as well to closed-source
> software.
:-) Did you ever tryed to check out OpenOffice.org code and have a look
at it and tryed to understand what goes where? Could be fun, but code
bloat is not an issue here. I think it's more about the ability to NOT
add features, the freedom to reduce and to leave things simple.
Basically, it's the freedom to focus. That is rarely the case in
OpenOffice.org. The complexity of OOo grew with its functionality/
features. Options and settings dialogs are the best example. The problem
here is, I think, that a lot of functionality finds its way into OOo
without being evaluated if it is relevant for a critical mass of
"ordinary" users, the 160 Millions who, unfortunately, are not on our
mailing lists and are not developers (See Joe Average from last OOo User
Survey). And that should never become an automatic process, although
this functionality might do great stuff. But it would unecessarily
increases the complexity of OOo.
>
>
> The section "Involving users" makes me think about IssueTracker
> statistics: if most users do not submit issues, can we say that most
> issues are submitted by developers, or is there a "long tail" effect
> and most issues are submitted by occasional submitters?
You got a point here. I think that the most issues, if they are not
submitted by developers, are submitted by community members that have
specific motivations doing so. And that are probably not the ordinary
OOo users since they rather have a motivation creating documents with
OOo instead of filing issues.
>
>
> Best Regards,
>
> Clément.
Many thanks, Clément for you comments.

Best,
Andreas
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

--
Sun Microsystems GmbH                Andreas Bartel
Nagelsweg 55                         User Experience Engineer
20097 Hamburg                        Phone: (+49 40)23646 672
Germany                              Fax:   (+49 40)23646 550
http://www.sun.com                   mailto:[hidden email]

Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028
Geschäftsführer: Thomas Schröder, Wolfgang Engels, Dr. Roland Bömer
Vorsitzender des Aufsichtsrates: Martin Häring


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Clément Pillias
Hello Andreas,

Le 3 févr. 09 à 21:16, Andreas Bartel a écrit :

> Clément Pillias schrieb:
>>
>> Unfortunately, the article is seven year old! Things have changed  
>> since then.
> I don't thing that time is an issue. I'd really like to know what  
> specifically has changed and to what extent.

The state of the art of usability management in OSS is outdated. The  
simple fact that we discuss it here should be a proof ;)
The biggest OSS projects now all have an UX team composed of paid  
professionals and benevolent ones. Some of these UX teams (such as  
Mozilla's) have even took a prominent role in marketing, contributing  
to establish a brand image.

>> Moreover, there are a lot of unproven assumptions. I particularly  
>> disagree with the section titled "Usability problems are harder to  
>> specify and distribute than functionality problems". I do it each  
>> time I discuss an issue here. And being a former programmer, I  
>> know that code has the same kind of issues than UI concerning  
>> "design inconsistency" and "major overhaul". But maybe I am an  
>> exception without knowing it, and breaking issues into independent  
>> parts is not a common task for UX specialists?
> The first sentence of this section "Functional problems are easier  
> to specify, evaluate and modularize than certain usability  
> problems." That said, I totally agree with the title of this  
> section. If you would really dig deep and start thinking about the  
> target user group, their tasks and their context in order to  
> specify a feature and how is should behave, the whole process  
> becomes much more complicated than creating a functional  
> specification when the requirements are known.

"When the requirements are known"? But "thinking about the target  
user group, their tasks and their context" is part of analyzing the  
requirements from an UX point of view. It is not part of designing a  
solution.

> I myself did also a lot of programming in Java, C/C++ and Matlab in  
> various distributed projects before I started at Sun. In contrast  
> to a quick and dirty UX solution, a piece of quick and dirty code  
> does not have to cause confusion by the user or a whole user group.

Except if it slows down further improvements because of bad design  
choices. But this is out of topic since you are discussing the  
effects of bad design, and not the complexity of the design itself.

> The second important point that the authors show in this section is  
> the involvement of many designers when they work autonomously and  
> thereby generate design inconsistencies. Not on purpose of course.

"when they work autonomously", ie, when they work on different parts  
of the interface without communication occurring between them to take  
global decisions. Luckily, this is not how OSS contributors work ;)

I want to highlight here one difference between OSS and CSS (although  
this difference tend to be reduced in CSS designed with agile  
methods) : UX designers take different roles. In particular the role  
of "reviewer" is very frequent in open-source software. Some people  
(like me?) do very little real design work, but take a lot of time to  
discuss solutions designed by other people. These people frequently  
serve as intermediaries between designers that would otherwise not  
realize that they should communicate.

> That assumption is supported by a journal paper (sorry, forgot the  
> title) that pointed out that even UX professionals that evaluate an  
> interface independently, very rarely find the same usability bugs  
> or generate similar solutions. Meaning, the overlap of uncovered  
> and solved UX problems is very small in expert reviews.

I really don't see the point of this paragraph… We could say the same  
thing about "functional problems": every programmer has is own  
approach and its own evaluation criterions. It does not make the  
evaluation process harder. I even see it as a good point for OSS:  
since more people can evaluate a design proposition, risks to miss a  
design error are reduced.

> Back to your question. Devide and conqure would not always work in  
> UX. Image you are designing a new way how to improve table handling  
> in OOo. You find a pritty cool solution for that rather "small"  
> problem.

This is not "divide and conquer": "divide and conquer" is a method to  
break a big problem into smaller related problems that can be solved  
independently.

> However, a user might like and learn the way you designer table  
> handling in Writer and would expect that tables in Impress, Base  
> and even Calc can be handled the same way. Or worse, a user might  
> expect that handling of other objects e.g. images, media etc. can  
> be done in an at least similar way. That would require to adapt  
> your solution to all the cases which suite user expectations. The  
> handling of "Insert" table rows and columns in Writer and Calc is a  
> good example for inconsistancy.

The problem here is a lack of generalization, not realizing that  
"table handling in Writer" is just a particular case of "table  
handling in OOo" which is itself just a particular case of "object  
handling in OOo". Here, typically, a good "divide and conquer"  
approach could have improved consistency if it had been done from an  
UX point-of-view.

"object handling in OOo" can be broken into related parts such as  
"inserting an object", "removing an object", "anchoring an object",  
"displacing an object", etc. who will all benefit from the  
independence of "creating a visual representation of the object".

What you describe is rather a consequence of solving issues without a  
global analyze than a consequence of some usability-specific  
difficulty. It happens also in closed-source software, but it is more  
evident in open-source software because of the "release often"  
policy. The solution is good documentation, which is something  
absolutely necessary to OSS but currently totally inexistent for UX.

>> The section "Open source projects lack the resources to undertake  
>> high quality usability work" assume that the only way to make  
>> "high quality usability work" is the way developed by and for  
>> closed-source software developers. I am opposed to that vision,  
>> and I think that, being open source, we have far more resources  
>> than closed-source software. But we still don't know how to use it  
>> efficiently :(
> Oooh, I very much see that point differently and I really would  
> like read what you have here in mind. UX methods are actually  
> independent of CSS or OSS, I think :-) However, first I would  
> really really ask you to give details here. Many thanks!

Well… it is such a big subject! Maybe later ;)

>> The section "OSS has an even greater tendency towards software  
>> bloat than commercial software" is simply wrong. Although the  
>> effects presented in this section are real, there are counter-
>> effects, such as the need to maintain and reuse code, or the peer  
>> pressure to make the code evolve (instead of adding another piece  
>> of code). And the paragraph about early adopters could apply as  
>> well to closed-source software.
> :-) Did you ever tryed to check out OpenOffice.org code and have a  
> look at it and tryed to understand what goes where? Could be fun,  
> but code bloat is not an issue here. I think it's more about the  
> ability to NOT add features, the freedom to reduce and to leave  
> things simple. Basically, it's the freedom to focus. That is rarely  
> the case in OpenOffice.org. The complexity of OOo grew with its  
> functionality/ features. Options and settings dialogs are the best  
> example. The problem here is, I think, that a lot of functionality  
> finds its way into OOo without being evaluated if it is relevant  
> for a critical mass of "ordinary" users, the 160 Millions who,  
> unfortunately, are not on our mailing lists and are not developers  
> (See Joe Average from last OOo User Survey). And that should never  
> become an automatic process, although this functionality might do  
> great stuff. But it would unecessarily increases the complexity of  
> OOo.

Have you seen Writer 2008 option pane? I believe that there is at  
least as many options in it than in OOo… Once again, everything in  
this paragraph could apply as well to closed-source software.

Clément.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Andreas Bartel
Hi Clément,

thanks for your comments. Let's continue :-)

Clément Pillias schrieb:

> Hello Andreas,
>
> Le 3 févr. 09 à 21:16, Andreas Bartel a écrit :
>
>> Clément Pillias schrieb:
>>>
>>> Unfortunately, the article is seven year old! Things have changed
>>> since then.
>> I don't thing that time is an issue. I'd really like to know what
>> specifically has changed and to what extent.
>
> The state of the art of usability management in OSS is outdated. The
> simple fact that we discuss it here should be a proof ;)
It can also be considered a proof that, possibly, only the two of us are
interested in that topic, since we're the only once who discuss ;-). I
hope I am wrong. Come Christopher, we are your two cents :-)
> The biggest OSS projects now all have an UX team composed of paid
> professionals and benevolent ones. Some of these UX teams (such as
> Mozilla's) have even took a prominent role in marketing, contributing
> to establish a brand image.
Partially agreed since that was discussed as a solution in the paper.

>
>>> Moreover, there are a lot of unproven assumptions. I particularly
>>> disagree with the section titled "Usability problems are harder to
>>> specify and distribute than functionality problems". I do it each
>>> time I discuss an issue here. And being a former programmer, I know
>>> that code has the same kind of issues than UI concerning "design
>>> inconsistency" and "major overhaul". But maybe I am an exception
>>> without knowing it, and breaking issues into independent parts is
>>> not a common task for UX specialists?
>> The first sentence of this section "Functional problems are easier to
>> specify, evaluate and modularize than certain usability problems."
>> That said, I totally agree with the title of this section. If you
>> would really dig deep and start thinking about the target user group,
>> their tasks and their context in order to specify a feature and how
>> is should behave, the whole process becomes much more complicated
>> than creating a functional specification when the requirements are
>> known.
>
> "When the requirements are known"? But "thinking about the target user
> group, their tasks and their context" is part of analyzing the
> requirements from an UX point of view. It is not part of designing a
> solution.
Could not disagree more. In User-Centered Design, you don't solely
analyze requirements, instead, users + their tasks + their contexts are
analyzed to generate requirements that the design process should obay
to. To decouple the design process from the analysis process would be a
strong vialation of the whole iterative UCD process which then would
make it difficult to create solutions that meet users' needs because
these were not considered during design.

>
>> I myself did also a lot of programming in Java, C/C++ and Matlab in
>> various distributed projects before I started at Sun. In contrast to
>> a quick and dirty UX solution, a piece of quick and dirty code does
>> not have to cause confusion by the user or a whole user group.
>
> Except if it slows down further improvements because of bad design
> choices. But this is out of topic since you are discussing the effects
> of bad design, and not the complexity of the design itself.
>
>> The second important point that the authors show in this section is
>> the involvement of many designers when they work autonomously and
>> thereby generate design inconsistencies. Not on purpose of course.
>
> "when they work autonomously", ie, when they work on different parts
> of the interface without communication occurring between them to take
> global decisions. Luckily, this is not how OSS contributors work ;)
>
> I want to highlight here one difference between OSS and CSS (although
> this difference tend to be reduced in CSS designed with agile methods)
> : UX designers take different roles. In particular the role of
> "reviewer" is very frequent in open-source software. Some people (like
> me?) do very little real design work, but take a lot of time to
> discuss solutions designed by other people. These people frequently
> serve as intermediaries between designers that would otherwise not
> realize that they should communicate.
I would not agree that in CSS development UX don't review things as
often as in OSS. Besides, review is just one important part of UX
engineering. Yes, I agree that in OSS lots of reviews are discussed by
many individuals. However, when each individual has her own needs in
mind or the needs of ordinary users are simply blurry or unknown, the
review process might take direction that are not desired or less optimal
to the target user group.

>
>> That assumption is supported by a journal paper (sorry, forgot the
>> title) that pointed out that even UX professionals that evaluate an
>> interface independently, very rarely find the same usability bugs or
>> generate similar solutions. Meaning, the overlap of uncovered and
>> solved UX problems is very small in expert reviews.
>
> I really don't see the point of this paragraph… We could say the same
> thing about "functional problems": every programmer has is own
> approach and its own evaluation criterions. It does not make the
> evaluation process harder. I even see it as a good point for OSS:
> since more people can evaluate a design proposition, risks to miss a
> design error are reduced.
Again, risks according to who's needs or requirements and design errors
in what standards? I have a strong feeling that it is frequently assumed
that everyone who participates in a review or a design process in OSS is
considered omniscient regarding the great mass of ordinary users who are
not involved in the community. My opinion here is that it's just not
true and even impossible without at least a small research activity.
This of course applies for UX in CSS but there, resouces are spend a lot
more often into requirements gathering in target user groups. Which is
quite hard as you see in your Renaissance project.

>
>> Back to your question. Devide and conqure would not always work in
>> UX. Image you are designing a new way how to improve table handling
>> in OOo. You find a pritty cool solution for that rather "small" problem.
>
> This is not "divide and conquer": "divide and conquer" is a method to
> break a big problem into smaller related problems that can be solved
> independently.
>
>> However, a user might like and learn the way you designer table
>> handling in Writer and would expect that tables in Impress, Base and
>> even Calc can be handled the same way. Or worse, a user might expect
>> that handling of other objects e.g. images, media etc. can be done in
>> an at least similar way. That would require to adapt your solution to
>> all the cases which suite user expectations. The handling of "Insert"
>> table rows and columns in Writer and Calc is a good example for
>> inconsistancy.
>
> The problem here is a lack of generalization, not realizing that
> "table handling in Writer" is just a particular case of "table
> handling in OOo" which is itself just a particular case of "object
> handling in OOo". Here, typically, a good "divide and conquer"
> approach could have improved consistency if it had been done from an
> UX point-of-view.
>
> "object handling in OOo" can be broken into related parts such as
> "inserting an object", "removing an object", "anchoring an object",
> "displacing an object", etc. who will all benefit from the  
> independence of "creating a visual representation of the object".
>
> What you describe is rather a consequence of solving issues without a
> global analyze than a consequence of some usability-specific
> difficulty. It happens also in closed-source software, but it is more
> evident in open-source software because of the "release often" policy.
> The solution is good documentation, which is something absolutely
> necessary to OSS but currently totally inexistent for UX.
>
>>> The section "Open source projects lack the resources to undertake
>>> high quality usability work" assume that the only way to make "high
>>> quality usability work" is the way developed by and for
>>> closed-source software developers. I am opposed to that vision, and
>>> I think that, being open source, we have far more resources than
>>> closed-source software. But we still don't know how to use it
>>> efficiently :(
>> Oooh, I very much see that point differently and I really would like
>> read what you have here in mind. UX methods are actually independent
>> of CSS or OSS, I think :-) However, first I would really really ask
>> you to give details here. Many thanks!
>
> Well… it is such a big subject! Maybe later ;)
>
>>> The section "OSS has an even greater tendency towards software bloat
>>> than commercial software" is simply wrong. Although the effects
>>> presented in this section are real, there are counter-effects, such
>>> as the need to maintain and reuse code, or the peer pressure to make
>>> the code evolve (instead of adding another piece of code). And the
>>> paragraph about early adopters could apply as well to closed-source
>>> software.
>> :-) Did you ever tryed to check out OpenOffice.org code and have a
>> look at it and tryed to understand what goes where? Could be fun, but
>> code bloat is not an issue here. I think it's more about the ability
>> to NOT add features, the freedom to reduce and to leave things
>> simple. Basically, it's the freedom to focus. That is rarely the case
>> in OpenOffice.org. The complexity of OOo grew with its functionality/
>> features. Options and settings dialogs are the best example. The
>> problem here is, I think, that a lot of functionality finds its way
>> into OOo without being evaluated if it is relevant for a critical
>> mass of "ordinary" users, the 160 Millions who, unfortunately, are
>> not on our mailing lists and are not developers (See Joe Average from
>> last OOo User Survey). And that should never become an automatic
>> process, although this functionality might do great stuff. But it
>> would unecessarily increases the complexity of OOo.
>
> Have you seen Writer 2008 option pane? I believe that there is at
> least as many options in it than in OOo… Once again, everything in
> this paragraph could apply as well to closed-source software.
I hope you mean Word 2008. I guess that the authors do not intend to
initiate some sort of a debate if CSS UX is better than OSS UX by saying
that certain UX failures never heppen in CSS. Instead, as they say it,
UX in CSS is a step ahead of OSS due to a higher maturity level of CSS
companies, a higher competitive pressure and some other organizational
factors.

More interesting would be question how we could improve? How can the UX
project of OOo profit from the insights in this paper, is there anything
we might change in the process? I think that it is an important aspect
of our community work.
>
>
> Clément.
Best,
Andreas
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

--
Sun Microsystems GmbH                Andreas Bartel
Nagelsweg 55                         User Experience Engineer
20097 Hamburg                        Phone: (+49 40)23646 672
Germany                              Fax:   (+49 40)23646 550
http://www.sun.com                   mailto:[hidden email]

Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028
Geschäftsführer: Thomas Schröder, Wolfgang Engels, Dr. Roland Bömer
Vorsitzender des Aufsichtsrates: Martin Häring


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Jaron Kuppers
Hi,

I find this discussion very interesting, but I don't have time to contribute
now.  Just thought you should know you're not alone.

Cheers,
Jaron
Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Andreas Bartel
Thanks Jaron, appreciate your comment!

Best,
Andreas

Am 04.02.2009 um 21:46 schrieb Jaron Kuppers:

> Hi,
>
> I find this discussion very interesting, but I don't have time to  
> contribute
> now.  Just thought you should know you're not alone.
>
> Cheers,
> Jaron


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Peter Junge
In reply to this post by Andreas Bartel
Hi,

Andreas Bartel wrote:
> Hello members,
>
> here is a link to a well elaborated article about usability in open
> source software.
>
> http://www.cs.waikato.ac.nz/~daven/docs/oss-wp.html

IMHO this article is not worth too much consideration, as it is about
six years old. Additionally it just plays around with several
misconceptions about FOSS, see paragraphs:
- Developers are not users
- Usability experts do not get involved in OSS projects
- Open source projects lack the resources to undertake high quality
usability work
- Commercial software establishes state of the art so that OSS can only
play catch-up

This all has significantly changed in the last couple of years.

While the bibliography is quite long, the paragraph 'OSS has an even
greater tendency towards software bloat than commercial software' comes
without any reference and is basically bla bla. No good academic work FMPOV.

[...]

Best regards,
Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Andreas Bartel
Hi Peter,

thanks for feedback. Some comments from an UX point of view:

Peter Junge schrieb:

> Hi,
>
> Andreas Bartel wrote:
>  
>> Hello members,
>>
>> here is a link to a well elaborated article about usability in open
>> source software.
>>
>> http://www.cs.waikato.ac.nz/~daven/docs/oss-wp.html
>>    
>
> IMHO this article is not worth too much consideration, as it is about
> six years old. Additionally it just plays around with several
> misconceptions about FOSS, see paragraphs:
> - Developers are not users
>  
trully not a misconception for OOo. Developers are not the target users
of e.g. OOo. Taking into account the results of our last user survey,
the dominant part of OOo users consists of people who are neither an OSS
community member nor a developer. In addition, it also would not make
sense to develop OOo for developers as the target audience only if you
want to grow in any educational or governmental sector.
> - Usability experts do not get involved in OSS projects
>  
Please correct me if I am wrong but the relation of coding pros in
contrast to UX professionals who are involved in OOo is rather
unbalanced, don't you thinl? I think there is a huge gap, or what are
the reasons we have a huge pile of UX bugs that are waiting for being fixed?
> - Open source projects lack the resources to undertake high quality
> usability work
>  
that is also true for OOo. Currently, even with support of Sun, we are
not able to conduct the research we actually planned to for the
Renaissance project because of the lack of resources. I can't say if
it's true for all other OSS project, which I don't think, however, it is
a current porblem in OOo.
> - Commercial software establishes state of the art so that OSS can only
> play catch-up
>  
Don't see the point here. UX methods such as the User-Centered Design,
improvement through iteration, Personas etc. were all developed in the
context of CSS long before they found their way, if at all, into OSS
development. Hence, we only can ans should profit from that development,
we can try to avoid the mistakes CSS projects did in the 80s and 90s,
from an UX perpective, e.g. put the ordinary user in the center of OOo
development and focus on her needs.
> This all has significantly changed in the last couple of years.
>
> While the bibliography is quite long, the paragraph 'OSS has an even
> greater tendency towards software bloat than commercial software' comes
>  
Yes, that is a potential threat, did you read their justifications for
that assumptions? I would partially agree with their argument because in
OSS development their is often a tendency to include solutions from
everyone for everyone who contributes. Which is good but from a product
management point of view, that is a bad habbit which can lead to a
product that doen't serve anyone because it lacks a focus on its
attenden user group. Consequently, you might find features that are
valuable but only for a small user group. That contributes to a
potential feature bloat.
> without any reference and is basically bla bla. No good academic work FMPOV.
>
> [...]
>
> Best regards,
> Peter
>
>  
Best,
Andreas
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>  


--
Sun Microsystems GmbH                Andreas Bartel
Nagelsweg 55                         User Experience Engineer
20097 Hamburg                        Phone: (+49 40)23646 672
Germany                              Fax:   (+49 40)23646 550
http://www.sun.com                   mailto:[hidden email]

Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028
Geschäftsführer: Thomas Schröder, Wolfgang Engels, Dr. Roland Bömer
Vorsitzender des Aufsichtsrates: Martin Häring


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Clément Pillias

Le 5 févr. 09 à 12:43, Andreas Bartel a écrit :

>> IMHO this article is not worth too much consideration, as it is  
>> about six years old. Additionally it just plays around with  
>> several misconceptions about FOSS, see paragraphs:
>> - Developers are not users

> trully not a misconception for OOo. Developers are not the target  
> users of e.g. OOo. Taking into account the results of our last user  
> survey, the dominant part of OOo users consists of people who are  
> neither an OSS community member nor a developer. In addition, it  
> also would not make sense to develop OOo for developers as the  
> target audience only if you want to grow in any educational or  
> governmental sector.

I am not sure about that. OOo is such a big project that the audience  
is very non-uniform, and even designing for the real user may be  
difficult for this project, reaching the limits of User-Centered Design.

Moreover, from what I have seen of issue handling, I suspect that:
* most UX issues are solved taking into account the suggestions from  
users that are not developers
* many developers code patches for features or parts of the  
application that they do not use, and so, it reduce the amount of  
misconceptions that they can have about how it should work.
* most developers are conscious that they are not in the targeted  
audience.

So, in my opinion, the assumption that "developers are not  
representative users" is not so important for OOo than for some other  
applications.

Clément.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Article: Usability of Open Source Software

Christoph Noack
In reply to this post by Andreas Bartel
Hi everyone,

I'm sorry that I join so late, especially since I'm very much interested
in solving/discussing some of the issues UX today suffers from. So
Clément, Andreas and the others - thank you so much for starting this
discussion.

Am Mittwoch, den 04.02.2009, 17:59 +0100 schrieb Andreas Bartel:

> Clément Pillias schrieb:
> > Le 3 févr. 09 à 21:16, Andreas Bartel a écrit :
> >> Clément Pillias schrieb:
> >>> Unfortunately, the article is seven year old! Things have changed
> >>> since then.
> >> I don't thing that time is an issue. I'd really like to know what
> >> specifically has changed and to what extent.
> >
> > The state of the art of usability management in OSS is outdated. The
> > simple fact that we discuss it here should be a proof ;)
> It can also be considered a proof that, possibly, only the two of us are
> interested in that topic, since we're the only once who discuss ;-). I
> hope I am wrong. Come Christopher, we are your two cents :-)

Before I start: In general, I do mostly agree with the observations and
conclusions in the paper. Unfortunately, some statements sound like
speculations and are not backed up by real data - but that is less
important to me, because the content itself looks rather valid to me.

Concerning the age, yes, this paper is about 7 years old. Does it
matter? If we look from the general OSS world, it really matters. All
other large scale OSS projects (e.g. Gnome, KDE, The Gimp, ...) did a
lot of work to improve their usability and sometimes even the User
Experience. If we look at them, they do not only catch up in usability,
they use their strengths to surpass their commercial counterparts. For
example, the trust in user feedback programs (collecting user data) may
be much higher for open-source projects. Have a look at [1].

[1] http://ingimp.org/

Does the age matter for OOo? In my opinion not, because the paper
outlines the situation for the volunteer part of community when the UX
project was initiated two years ago. (Why is the term 'volunteer'
important here: there was usability testing and UX work at Sun, but even
the work was not accessible to the public.)

So if we discuss how "valid" the situation in the paper is today for
OOo, we should look how we progressed in the last two years. In my point
of view, in this big community (volunteers, commercial interests,
organisations, ...), things are much much slower than I would have
thought. Finally, from my point of view, the paper is still valid for
our community.

And here I will stop and wait what you think about it...

> > The biggest OSS projects now all have an UX team composed of paid
> > professionals and benevolent ones. Some of these UX teams (such as
> > Mozilla's) have even took a prominent role in marketing, contributing
> > to establish a brand image.
> Partially agreed since that was discussed as a solution in the paper.

Yep, and if you look at it in detail, the UX teams of those projects
consist of very few well trained people who work closely together. That
automatically reduces the variance of decisions/opinions and helps the
team being trusted. Due to the "publicity" of the team members, it is
easier for the environment (e.g. the developers) to be sure about the
quality of the outcome.

Something we should think about?

> >>> Moreover, there are a lot of unproven assumptions. I particularly
> >>> disagree with the section titled "Usability problems are harder to
> >>> specify and distribute than functionality problems". I do it each
> >>> time I discuss an issue here. And being a former programmer, I know
> >>> that code has the same kind of issues than UI concerning "design
> >>> inconsistency" and "major overhaul". But maybe I am an exception
> >>> without knowing it, and breaking issues into independent parts is
> >>> not a common task for UX specialists?
> >> The first sentence of this section "Functional problems are easier to
> >> specify, evaluate and modularize than certain usability problems."
> >> That said, I totally agree with the title of this section. If you
> >> would really dig deep and start thinking about the target user group,
> >> their tasks and their context in order to specify a feature and how
> >> is should behave, the whole process becomes much more complicated
> >> than creating a functional specification when the requirements are
> >> known.
> >
> > "When the requirements are known"? But "thinking about the target user
> > group, their tasks and their context" is part of analyzing the
> > requirements from an UX point of view. It is not part of designing a
> > solution.
> Could not disagree more. In User-Centered Design, you don't solely
> analyze requirements, instead, users + their tasks + their contexts are
> analyzed to generate requirements that the design process should obay
> to. To decouple the design process from the analysis process would be a
> strong vialation of the whole iterative UCD process which then would
> make it difficult to create solutions that meet users' needs because
> these were not considered during design.

My fingers want to start typing "yes, no, maybe, somehow" at the same
time :-)

The initial statement "Usability problems are harder to specify and
distribute than functionality problems" has caused different opinions...
Reading your statements, it seems that it depends on the given knowledge
(the constraints, the requirements) whether functionality problems or UX
problems are harder to work on.

>From a developer's point-of-view, e.g. the re-factoring of software (for
better extendability) or improving the performance requires to look at
the whole software structure - at this point I disagree somehow with the
statement in the paper.

The difference are the constraints, the knowledge of the problem. Let's
assume that the developer develops for a platform which is rather new to
him, it's different to the other ones:
      * The platform runs differently on each computer (the same command
        is interpreted differently)
      * The platform is non-deterministic (sometimes things need more
        time than others)
      * The computer's performance varies dependent on energy quality
        (e.g. wind energy, solar energy, nuclear energy)
      * The computer's performance varies dependent on the weather
        outside (sometimes skipping some of the commands to be
        performed)
      * The platform may decide to just stop a program, because an
        algorithm decides that other things are more important
      * The platform's performance decreases over time

Now, please ask a developer to write a program which runs as smooth as
possible. At this point, UX and software development is very similar in
being complex and complicated. The quality of the solution will strongly
depend on what you know and how good that information is. And usually,
software developers have advantages here... (technical platforms are
specified and don't vary).

Finally, in my opinion it is much harder to work on UX/usability
problems, because of people having different personal profiles,
different contexts (private, professional), different knowledge,
different abilities (color blindness), ... The main work of UX is to
consider these additional information when creating a solution. I think
that is what Andreas had in mind.

[...]

> >> The second important point that the authors show in this section is
> >> the involvement of many designers when they work autonomously and
> >> thereby generate design inconsistencies. Not on purpose of course.
> >
> > "when they work autonomously", ie, when they work on different parts
> > of the interface without communication occurring between them to take
> > global decisions. Luckily, this is not how OSS contributors work ;)
> >
> > I want to highlight here one difference between OSS and CSS (although
> > this difference tend to be reduced in CSS designed with agile methods)
> > : UX designers take different roles. In particular the role of
> > "reviewer" is very frequent in open-source software. Some people (like
> > me?) do very little real design work, but take a lot of time to
> > discuss solutions designed by other people. These people frequently
> > serve as intermediaries between designers that would otherwise not
> > realize that they should communicate.
> I would not agree that in CSS development UX don't review things as
> often as in OSS. Besides, review is just one important part of UX
> engineering. Yes, I agree that in OSS lots of reviews are discussed by
> many individuals. However, when each individual has her own needs in
> mind or the needs of ordinary users are simply blurry or unknown, the
> review process might take direction that are not desired or less optimal
> to the target user group.

I fully agree, especially since a reviews sometime require that the
reviewer itself knows how to design. Therefore I can only invite people
to propose designs which can be reviewed here. Some people might find
this useless to them, but the increase in knowledge is enormous (my
opinion). It strongly helps to analyze the own personal strengths and to
move from an "this is my personal opinion" towards a "this might cause
issues for the intended user-base".

> >> That assumption is supported by a journal paper (sorry, forgot the
> >> title) that pointed out that even UX professionals that evaluate an
> >> interface independently, very rarely find the same usability bugs or
> >> generate similar solutions. Meaning, the overlap of uncovered and
> >> solved UX problems is very small in expert reviews.
> >
> > I really don't see the point of this paragraph… We could say the same
> > thing about "functional problems": every programmer has is own
> > approach and its own evaluation criterions. It does not make the
> > evaluation process harder. I even see it as a good point for OSS:
> > since more people can evaluate a design proposition, risks to miss a
> > design error are reduced.

The difference is, that in software development it is rather easy to
establish *and* verify quality criteria. Either it works on a given
machine, or it does not. For UX, invite 100 people and test their
reactions - then you can be somehow sure that it will work for most
people. (Is it what you meant, Andreas?)

Clemént, as a consequence, I think it is important for the UX team that
everyone develops certain skills concerning UX - then it should be
possible to provide substantiate feedback (not personal judgments), even
with different focus.

[...]

> >>> The section "Open source projects lack the resources to undertake
> >>> high quality usability work" assume that the only way to make "high
> >>> quality usability work" is the way developed by and for
> >>> closed-source software developers. I am opposed to that vision, and
> >>> I think that, being open source, we have far more resources than
> >>> closed-source software. But we still don't know how to use it
> >>> efficiently :(
> >> Oooh, I very much see that point differently and I really would like
> >> read what you have here in mind. UX methods are actually independent
> >> of CSS or OSS, I think :-) However, first I would really really ask
> >> you to give details here. Many thanks!
> >
> > Well… it is such a big subject! Maybe later ;)

I'm looking forward :-)

> >>> The section "OSS has an even greater tendency towards software bloat
> >>> than commercial software" is simply wrong. Although the effects
> >>> presented in this section are real, there are counter-effects, such
> >>> as the need to maintain and reuse code, or the peer pressure to make
> >>> the code evolve (instead of adding another piece of code). And the
> >>> paragraph about early adopters could apply as well to closed-source
> >>> software.
> >> :-) Did you ever tryed to check out OpenOffice.org code and have a
> >> look at it and tryed to understand what goes where? Could be fun, but
> >> code bloat is not an issue here. I think it's more about the ability
> >> to NOT add features, the freedom to reduce and to leave things
> >> simple. Basically, it's the freedom to focus. That is rarely the case
> >> in OpenOffice.org. The complexity of OOo grew with its functionality/
> >> features. Options and settings dialogs are the best example. The
> >> problem here is, I think, that a lot of functionality finds its way
> >> into OOo without being evaluated if it is relevant for a critical
> >> mass of "ordinary" users, the 160 Millions who, unfortunately, are
> >> not on our mailing lists and are not developers (See Joe Average from
> >> last OOo User Survey). And that should never become an automatic
> >> process, although this functionality might do great stuff. But it
> >> would unecessarily increases the complexity of OOo.
> >
> > Have you seen Writer 2008 option pane? I believe that there is at
> > least as many options in it than in OOo… Once again, everything in
> > this paragraph could apply as well to closed-source software.
> I hope you mean Word 2008. I guess that the authors do not intend to
> initiate some sort of a debate if CSS UX is better than OSS UX by saying
> that certain UX failures never heppen in CSS. Instead, as they say it,
> UX in CSS is a step ahead of OSS due to a higher maturity level of CSS
> companies, a higher competitive pressure and some other organizational
> factors.

Mmh, I don't understand that section. Does it mean that CSS UX lost,
because of the many options in Word 2008? Maybe it is a result of UX won
at Microsoft, what would have been the alternative? :-)

The discussion of "feature are added without being evaluated" sounds
funny, because companies like Sun seem to rely on paying customers who
strongly desire this or that in StarOffice before they start adopting
it. Sometimes this is very good (the community gets strongly desired
functionality), sometimes it is less positive (because marketing and
sales may focus on the requires of a single customer).

> More interesting would be question how we could improve? How can the UX
> project of OOo profit from the insights in this paper, is there anything
> we might change in the process? I think that it is an important aspect
> of our community work.

Yep, there are many things which seem to be important for both a vivid
and effective UX community work. But at the moment, I just want to know
what you think...


Andreas, Clément, thank you very much for your effort!

Christoph


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]